Sign inSign up

sslhep/ml_front

Sponsored OSS

By University of Chicago

•Updated about 4 years ago

ML services on K8s

Image
Languages & frameworks
Machine learning & AI
Data science
0

10K+

sslhep/ml_front repository overview

⁠ML front

Exposes k8s ml resources to users as services:

  • JupyterLab.
  • TensorFlowAsAService (soon)

A deployment creates node.js based web frontend. The front end authenticates users through globus. User can select to instantiate several different services, select hardware needed, duration, and other parameters.

⁠Configuration options

VariableMeaningExample value
EVENTThis will be used in mails, kibana dashboardscodas2019
SITENAMEaddress where the front end will be accessible fromml.maniac.uchicago.edu
STATIC_PATHdirectory containing static site customizationsml-usatlas-org,
NAMESPACEk8s namespace where all of pods and services will runmaniac-ml
TITLEWill be shown in web browser title bar
SSLAre JupyterLabs served on https?false
APPROVAL_REQUIREDIs authorization required (on top of globus authentication)?true
APPROVAL_EMAILOnly if authorization is required[email protected]⁠
SINGLE_INSTANCELimit to only one private Jupyter instancefalse
PUBLIC_INSTANCEExpose a public JupyterLab instance that anyone can usetrue
MONITOREnable user access to monitoring infotrue
REMOTE_K8SEnable non local k8s service spawnfalse
REPORTINGAre pods monitored in Elasticsearctrue
JL_PODUsed to customize JuputerLab pod./jupyter-pod.json
JL_SERVICEUsed to customize JuputerLab service./jupyter-service.json
PLUGINSlist of enabled plugins["MONITOR", "SPARK", "PUBLIC_INSTANCE"]

⁠Authorizing users

If configured to require authorization (APPROVAL_REQUIRED), authorization is automatically sought for authenticated users. Authorization request is approved by a platform owner.

⁠To look up users check kibana page:

http://atlas-kibana.mwt2.org/goto/7ad9cbf9627d180d24e06ac018ac6c4a⁠

⁠TODO

  • switch from using .json files to fully generated objects.
  • completely move to ingress controllers. (spark too)
  • change how upper left corner Title is set (should be in pug.)
  • add shared filesystem
  • add option to mount CVMFS
  • add option to update some service parameters
  • add cloud deployments
  • add postfixes to pods/services
  • add ml-front pods monitoring info collection
  • SPARK submission
    • submission page
      • file upload (multiple files?)
      • n executors
      • executor memory
      • job name
    • results page. status, link to output.
  • write documentation
  • TFAAS
  • web site
    • Services - running/terminated service
    • Services - services title

Tag summary

Content type

Image

Digest

Size

379.3 MB

Last updated

about 4 years ago

docker pull sslhep/ml_front

This week's pulls

Pulls:

382

Last week