How to install new libraries with Spark interpreter
To install new libraries you have to follow the following steps:
1. Go to manage interpreters
With an administrator user, in the Notebook screen, in the upper-right corner, go to MANAGE INTERPRETER menu.
2. Search for Spark interpreter
Search for Spark interpreter to change its configuration.
3. Edit configuration to add new dependencies
Once you find the Spark interpreter, then click on edit button and scroll down.
At the end, you will be able to add new dependencies in the following way. In this example we add some common dependencies:
Libraries | Reference |
---|---|
Twitter Streaming | org.apache.bahir:spark-streaming-twitter_2.11:2.0.1 |
MongoDB | org.mongodb.spark:mongo-spark-connector_2.11:2.3.0 |
MySQL | mysql:mysql-connector-java:jar:8.0.16 |
Once configured, you can SAVE the configuration. It should restart the interpreter, so next time you exectute a Spark paragraph changes will be availables. If not, try to restart interpreter manually from menu showed in step 1.
How to install manually from external jars
If you want to add new, external jars, you can download those manually and install them inside the Notebook container.
For example: mongodb driver.
From correct folder (/zeppelin/local-repo/spark), download the jar (from other repo, internet, etc.)
It is also necessary to add the artifact in the Zeppelin configuration (same as step 3). This is required because Zeppelin creates an internal config file with all its dependencies.
Finallly, you’ll need to reload the interpreter.