elyra/kernel-spark-py repository overview

This image enables the use of an IPython kernel launched from Jupyter Enterprise Gateway⁠ within a Kubernetes cluster. It is built on the base image elyra/kernel-py⁠, and adds Apache Spark 2.4.6⁠. Note: The ability to use the kernel within Spark within a Docker Swarm configuration probably won't yield the expected results.

⁠What it Gives You

• IPython kernel support (with debugger)
• Data science libraries⁠
• Spark on kubernetes support from within a Jupyter Notebook

⁠Basic Use

Deploy enterprise-gateway⁠ per its instructions and configured to the appropriate environment.

Launch a gateway-enabled Jupyter Notebook application against the Enterprise Gateway instance and pick the desired kernel to use in your notebook.

For more information, check our repo⁠ and docs⁠.