Soleil: Lucid Configurations#

Soleil is a Python object configuration mechanism inspired by Meta’s Hydra. It’s main goal is to provide a Python mechanism for machine learning researchers to assemble flexible experiments in a way that is declarative, inheritable, automatically CLI-invokable and intuitively overridable.

Taming the Curse of main.py#

Machine learning experiments are often complex systems where the different components (model, optimizer, learning rate scheduler, dataset, dataloader, transformations/augmentations, and visualizations) have multiple options each defined by many different parameters.

Usually, all of these components are hard-coded into a single main.py file with an argparse command line interface that uses a global parameter namespace. Together, this main.py/argparse recipe becomes unwieldy, difficult to extend and error-prone, slowing down further research development.

Soleil’s Approach#

Soleil’s approach is comprised of the following:

Familiar Python Syntax

Soleil configuration files (*.solconf files) use familiar Python syntax and can describe any installed function call, Python object, or composition thereof.

Inheritable Object Descriptions

The description of an object instance or function call in a configuration file takes the form of a class declaration – object descriptions hence benefit from standard Python inheritance mechanisms. Class declarations can be nested to support descriptions of input parameters.

Portable Package Organization

Soleil configuration files can be organized into portable, Python-like packages. Sub-packages can be made to correspond to system components (e.g., models) that can be easily swapped by loading (see load() and submodule()).

Automatic, Flexible CLI

Any Soleil configuration file can be run as an executable directly using the built-in solex tool – a fully fledged, extensible command line utility that plays nicely with the builtin argparse module. Usage as a decorator (see @solex) or within a standard argparse parser is also supported (See Automatic CLI).

Sandboxed Overridability

Soleil-enabled CLI calls involving soleil-configured objects can be modified or overriden from the command line using a subset of the Python syntax that is sandboxed through AST magic.

Warning

Soleil configurations are in effect Python programs – only run configurations that you trust.

Note

The Soleil logo and other images in this documentation were produced using a free version of DALL-E.

Indices#