The craceplot package#

Maintainers: Yunshuang XIAO

Contact: http://github/to/craceplot

Introduction#

The craceplot (named: cplot) package provides different plots to visualize the data generated by the crace software for automatic algorithm configuration (hyper-parameter optimization).

This package provides visualizations of:

  • Parameter configurations using parallel coordinates with param_parallelcoord() or param_parallelcat().

  • Configurations performance (training and testing) with qual_boxplot(), qual_heatmap() and qual_scatter().

  • Sampling distributions with param_boxplot(), param_histplot(), param_pairplot(), param_heatmap(), param_jointplot() and param_sunburst().

For more details about these functions, please check the user guide of the package and the documentation of the functions implemented in the package.

The aim of this package is to provide support for the analysis of the single crace run and its best parameter settings found, the assessment of the parameter space explored by crace and the overall performance of the configuration process.

Keywords: automatic configuration, offline tuning, parameter tuning, parameter visualization, crace.

Requisites#

Python is required for running crace and to use craceplot.

User guide#

A user guide comes with the package. The following is a quick-start guide. The user guide gives more detailed instructions.

Installing Python#

This section gives a quick Python installation guide that will work in most cases. The official instructions are available at official python site.

GNU/Linux#

You should install Python from your package manager. On a Debian/Ubuntu system it will be something like

sudo apt-get install python3 python3-dev

Once Python is installed, you can use conda, pip or the source code to install the crace package. See instructions below.

OS X#

For macOS 10.9 (Jaguar) up until 12.3 (Catalina) the operating system includes Python 2, which is no longer supported and is not a good choice for development. You should go to do the downloads page and download the installer.

For newer versions of macOS, Python is no longer included by default and you will have to download and install it. You can refer to the Python documentation for more details on the installation process and getting started.

You can install Python directly from a PyPI mirror (https://pypi.org/). Alternatively, you can just brew the Python formula from the science tap (unfortunately it does not come already bottled so you need to have Xcode (https://developer.apple.com/xcode/download/) installed to compile it):

# install homebrew if you don’t have it
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

# install python 3
brew tap homebrew/science
brew install python

Once Python is installed, you can use conda, pip or the source code to install the craceplot package.

Windows#

You can install Python from the official website (https://www.python.org/downloads/windows). We recommend that you install Python on a filesystem path without spaces, special characters or long names, such as C:\Python.

Note that Python 3.5 - 3.8 cannot be used on Windows XP or earlier and Python 3.9 - 3.13 cannot be used on Windows 7 or earlier.

Once Python is installed, you can use conda, pip or the source code to install the craceplot package.

Installing craceplot#

Requirements#

  • Python (>= 3.6.0) is required for running craceplot package.

  • crace (>= 0.0.0) is required for running craceplot package, and you must have completed log files from one crace run in advance. See crace user guide for installing crace and obtaining log files.

The craceplot package can be installed automatically with conda, with pip or from source. We advise to use the automatic installation unless particular circumstances do not allow it. The instructions to install cplot with the three mentioned methods are the following:

Install automatically with conda#

If you use Conda, execute the following line at the shell to install the package:

# Use the environment where you install crace
conda activate my-env
# If you want to install from conda-forge
conda config --env --add channels conda-forge
# The actual install command
conda install cplot

Alternatively, within the Conda graphical interface (Anaconda Navigator), you may search and install cplot.

Install automatically with pip#

If you use Pip, execute the following line at the shell to install the package:

pip3 install cplot

Also when using pip, it’s good practice to use a virtual environment and here is the guide for using the virtual environment.

Manual download and installation from source#

If the previous installation instructions fail because of insufficient permissions and you do not have sufficient admin rights to install cplot system-wide, then you need to force a local installation. From the cplot package PyPI website (https://pypi.org/), search and download one of the three versions available depending on your operating system:

  • cplot_cplotversion.tar.gz (Unix/Linux/BSD/macOS)

  • cplot_cplotversion.tar (Unix/Linux/BSD/macOS)

  • cplot_cplotversion.zip (Windows/macOS)

  • cplot-cplotversion-py3-none-any.whl (python3, any platform)

Alternatively, you can use wget or curl from the command line to download the file:

# PyPI
wget https://files.pythonhosted.org/packages/source/p/package_name /package_name-version.tar.gz
# Github
wget https://github/cplot

To install the package on GNU/Linux, macOS or Windows using the formats .tar.gz, .tar or .zip, you should first extract the source files and then navigate to the directory containing the extracted files and run the following command to install the package:

python3 setup.py install

To install the package on any platform using the Wheel format, you should run the following command to install the package:

pip3 install /path/to/cplot-@cplotversion@-py3-none-any.whl

Testing the installation and invoking cplot#

Once cplot has been installed, list by using command line at the shell

# List installed packages at the shell
pip3 list
# Show the information of cplot at the shell
pip3 show cplot

or load the package and test that the installation was successful by opening a Python console and executing:

# Load and test at the Python console
import cplot
print(cplot.__version__)

Checking the installation path of cplot#

To check the installation path of cplot using command line at the shell:

which cplot

The output of this line must be $PYTHON_HOME/bin/cplot, which means the path before /bin/cplot is $PYTHON_HOME. Here, $CPLOT_HOME should be:

CPLOT_HOME=$PYTHON_HOME/lib/python3.X/site-packages/cplot

Also, you can check the installation path of cplot opening a Python console and executing:

import importlib.util
print(importlib.util.find_spec('cplot').submodule_search_locations)

This command must print out the filesystem path where cplot is installed. In the remainder of this guide, the variable $CPLOT_HOME is used to denote this path. When executing any provided command that includes the $CPLOT_HOME variable, do not forget to replace it with the true installation path of cplot.

On GNU/Linux or macOS, you can let the operating system know where to find cplot by defining the $PYTHON_HOME variable and adding it to the system $PATH. Append the following commands to ~/.bash_profile, ~/.bashrc or ~/.profile:

# Replace <CPLOT_HOME> with the cplot installation path
export CPLOT_HOME=<CPLOT_HOME>
# Tell operating system where to find cplot
export PATH=${PYTHON_HOME}/bin/:$PATH

Then, open a new terminal and launch cplot as follows:

cplot --help

Alternatively, you may directly invoke cplot within the Python console by executing:

from cplot import start_cplot
start_cplot("--help")
Show Source