Skip to content

Dockerize the PA pipeline #463

@pauldhein

Description

@pauldhein

Overview

We need to create a docker container that can run the full translation process from Fortran source to GrFN. As a first step in that process we need to create a docker container that will handle running the PA pipeline. @hlim1 I am assigning this task to you but feel free to ask me and @pratikbhd for assistance as needed.

@skdebray I am adding you to this issue so that you will be kept up-to-date about our teams progress.

Task Goal

Create a minimal docker container that can run the program analysis pipeline.

Implementation Details

  • Make all changes needed for this task on the dssat_pet branch
  • Locate the Dockerfile for the container in: delphi/translators/for2py
  • Use the Dockerfile provided below as a stub for the new Dockerfile
    • Remove dependencies that are unnecessary for for2py
    • Change the last few lines to use delphi (for now) and set the WORKDIR to be ready for script execution
  • Have the docker container execute the script: scripts/f2grfn_standalone.py
  • Use PETPT.for as the test argument while making the container
  • When the docker container is run via the CMD it should generate an _AIR.JSON file for the input fortran module and return that to the directory on the on the source machine where the docker container was launched from

Dockerfile Stub

FROM        ubuntu:19.10
MAINTAINER  Paul D. Hein <pauldhein@email.arizona.edu>
CMD         bash

# ==============================================================================
# INSTALL SOFTWARE VIA THE UBUNTU PACKAGE MANAGER
# ==============================================================================
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update
RUN apt-get -y --no-install-recommends install \
  git \
  curl \
  build-essential \
  gcc \
  pkg-config \
  python3 \
  python3-dev \
  python3-pip \
  openjdk-8-jdk \
  antlr4 \
  graphviz \
  libgraphviz-dev \
  doxygen

RUN git config --global user.email "pauldhein@email.arizona.edu"
RUN git config --global user.name "Paul Hein"
# ==============================================================================

# ==============================================================================
# INSTALL PYTHON PACKAGES VIA PIP
# NOTE: Packages are installed in stages to avoid memory errors on DockerHub
# ==============================================================================
# Upgrading setuptools and installing wheel
RUN pip3 install --upgrade setuptools
RUN pip3 install wheel

# Common-use and testing packages
RUN pip3 install \
  antlr4-python3-runtime==4.8 \
  pytest==5.4.1 \
  pytest-cov==2.8.1 \
  Pygments==2.3.1 \
  tqdm==4.29.0

# Adding PyTorch separately due to memory constraints
RUN pip3 install \
  torch==1.4.0+cpu \
  -f https://download.pytorch.org/whl/torch_stable.html

# Scientific computing packages
RUN pip3 install \
  networkx==2.4 \
  numpy==1.18.2 \
  pandas==1.0.3 \
  sympy==1.5.1 \
  scikit_learn==0.22.2.post1 \
  SALib==1.3.8 \
  nltk==3.4.5 \
  torchtext==0.5.0

# Python Visualization libraries
RUN pip3 install \
  pygraphviz==1.5 \
  matplotlib==3.2.1 \
  plotly==4.5.4 \
  seaborn==0.10.0

# Web framework packages
RUN pip3 install \
  Flask==1.1.1 \
  flask_codemirror==1.1 \
  flask_wtf==0.14.3 \
  WTForms==2.2.1

# Unassigned other stuff
RUN pip3 install SPARQLWrapper==1.8.5
# ==============================================================================

# ==============================================================================
# SETUP THE AUTOMATES REPOSITORY AND ENVIRONMENT
# ==============================================================================
RUN git clone https://github.com/ml4ai/automates
ENV PYTHONPATH="/automates/src:$PYTHONPATH"
WORKDIR /automates
# ==============================================================================

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions