Python PySpark Training Repository ============== **Author:** *Yûki VACHOT* **Updated:** **10/01/24** # CONTENT TABLE --- # Installation `python -m venv ` - [Python 3.11.7](https://www.python.org/downloads/) - [Spark 3.5.0 with Hadoop 3.0.0](https://spark.apache.org/downloads.html) - [winutils.exe, .pdb and hadoop.dll](https://github.com/steveloughran/winutils/tree/master/hadoop-3.0.0/bin) - [Java JDK 17](https://www.azul.com/downloads/?version=java-17-lts&package=jdk#zulu) --- # Run Python PySpark - `python init.py` --- # Run Python Test - path from src/test_pyspark_training - `pytest -k test_` --- # Run pylint for code check --- # Run Python doc with Sphinx ---