15 lines
No EOL
528 B
Markdown
15 lines
No EOL
528 B
Markdown
# Python PySpark Training Repository
|
|
|
|
## Installation
|
|
- [Python 3.10](https://www.python.org/downloads/)
|
|
- pyspark=3.1.1
|
|
- findspark
|
|
- pyspark-test
|
|
- [Spark 3.1.1](https://spark.apache.org/downloads.html)
|
|
- [Hadoop 3.3.6](https://hadoop.apache.org/releases.html)
|
|
- [Java JDK 11](https://www.oracle.com/java/technologies/downloads/#java11)
|
|
- (not mandatory) [Anaconda for conda](https://www.anaconda.com/download/)
|
|
|
|
## Run Python Test
|
|
- path from src/test_pyspark_training
|
|
- `pytest -k test_` |