How to free up memory in the Notebooks?
- DevPortal Editor (Unlicensed)
This tutorial will explain how to free up memory in the software supporting the Notebooks: Zeppelin.
Before following this tutorial, you must understand the configuration of the Notebooks (available in the tutorial Interpreter configuration in the Notebooks module).
As the previous tutorial explains, there are three possibilities for configuring the interpreters:
- Globally: An interpreter process for all the Notebooks.
- Per Note: An interpreter process for each Notebook.
- Per User: An interpreter process for each Zeppelin user (other than OnesaitPlatform user).
Some interpreters, such as Spark, use a minimum RAM per process of 1 GB so, when it is configured as Per Note, this will imply that there will be as many processes as Notebooks executed.
Below is an example table:
| Executed Notebooks | Interpreter configuration | Minimum interpreter memory (GB) | Minimum total consumed memory (GB) | Description |
|---|---|---|---|---|
| 1 | Globally | 1 | 1 | 1 Notebook uses 1 interpreter process (variables are shared between Notebooks) |
| 2 | Globally | 1 | 1 | 2 Notebooks use 1 interpreter process (variables are shared between Notebooks) |
| 3 | Globally | 1 | 1 | 3 Notebooks use 1 interpreter process (variables are shared between Notebooks) |
| 1 | Per Note | 1 | 1 | 1 Notebook uses 1 interpreter process (variables are NOT shared between Notebooks) |
| 2 | Per Note | 1 | 2 | 2 Notebooks use 2 interpreter processes (variables are NOT shared between Notebooks) |
| 3 | Per Note | 1 | 3 | 3 Notebooks use 3 interpreter processes (variables are NOT shared between Notebooks) |
Each interpreter process starts with the first execution of a paragraph of that interpreter, but the process does not stop when the last paragraph is executed. This allows the data to be available whenever the Notebook is opened.
This situation can cause a large memory consumption when you run a Notebook that you do not want to keep the data in memory.
The correct way to end that interpreter process (as long the configuration is Per Note) is by doing a Restart of the interpreter. If a Restart with a Globally configuration is performed, the interpreter will be stopped in all Notebooks, interrupting its execution or availability.
That way, once you have finished with the execution of a Notebook and you do not want to keep it running, you can finish the interpreter process and free your memory.
Below is the Restart Interpreter option from a Notebook.
