You can manipulate your configuration with overrides (via the command line or the Compose API). This includes:
Modifying the Defaults ListModifying the config objectOverrides matching a config group are modifying the Defaults List;The rest are manipulating the config object.
Basic examplesModifying the Config ObjectOverriding a config value : foo.bar=valueAppending a config value : +foo.bar=valueAppending or overriding a config value : ++foo.bar=valueRemoving a config value : ~foo.bar, ~foo.bar=valueModifying the Defaults ListOverriding selected Option: db=mysql, server/db=mysqlAppending to Defaults List: +db=mysql, +server/db=mysqlDeleting from Defaults List: ~db, ~db=mysql, ~server/db, ~server/db=mysqlGrammarHydra supports a rich DSL in the command line.Below are the parser rules from grammar. You can see the full Lexer and Parser definitions on GitHub.
OverrideParser.g4// High-level command-line override.override: ( key EQUAL value?// key=value, key= (for empty value)| TILDE key (EQUAL value?)? // ~key | ~key=value| PLUS PLUS? key EQUAL value?// +key= | +key=value | ++key=value) EOF;// Key:key : packageOrGroup (AT package)?; // key | group@pkgpackageOrGroup: package | ID (SLASH ID)+;// db, hydra/launcherpackage: ( | ID | KEY_SPECIAL | DOT_PATH);// db, $db, hydra.launcher, or the empty (for _global_ package)// Elements (that may be swept over).value: element | simpleChoiceSweep;element: primitive| listContainer| dictContainer| function;simpleChoiceSweep: element (COMMA element)+// value1,value2,value3;// Functions.argName: ID EQUAL;function: ID POPEN (argName? element (COMMA argName? element )* )? PCLOSE;// Data structures.listContainer: BRACKET_OPEN // [], [1,2,3], [a,b,[1,2]](element(COMMA element)*)?BRACKET_CLOSE;dictContainer: BRACE_OPEN (dictKeyValuePair (COMMA dictKeyValuePair)*)? BRACE_CLOSE; // {}, {a:10,b:20}dictKeyValuePair: dictKey COLON element;// Primitive types.primitive: QUOTED_VALUE// 'hello world', "hello world"| (ID // foo_10| NULL// null, NULL| INT// 0, 10, -20, 1_000_000| FLOAT // 3.14, -20.0, 1e-1, -10e3| BOOL// true, TrUe, false, False| INTERPOLATION // ${foo.bar}, ${oc.env:USER,me}| UNQUOTED_CHAR // /, -, \, +, ., $, %, *, @, ?, || COLON // :| ESC// \\, \(, \), \[, \], \{, \}, \:, \=, \ , \\t, \,| WS // whitespaces)+;// Same as `primitive` except that `COLON` and `INTERPOLATION` are not allowed.dictKey:(ID // foo_10 | NULL// null, NULL | INT// 0, 10, -20, 1_000_000 | FLOAT // 3.14, -20.0, 1e-1, -10e3 | BOOL// true, TrUe, false, False | UNQUOTED_CHAR // /, -, \, +, ., $, %, *, @, ?, | | ESC// \\, \(, \), \[, \], \{, \}, \:, \=, \ , \\t, \, | WS // whitespaces)+;ElementsKeyKey is the component before the =. A few examples:
foo.bar# A config keyhydra/launcher# A config groupgroup@pkg # A config group assigned to the package pkggroup@pkg1:pkg2# A config group changing the package from pkg1 to pkg2Quoted valuesHydra supports both double quotes and single quoted values.Quoted strings can accept any value between the quotes, but some characters need escaping:
to include a single quote in a single quoted string, use \' (for double quotes in a double quoted string, use \")any sequence of \ characters preceding a quote (either an escaped quote as described in the previous point, or the closing quote)must be escaped by doubling the number of \Double quotes"hello there""escaped \"double quote\"""the path is C:\\\"some folder\"\\""1,2,3""{a:10} ${xyz}""'single quoted string'"Single quotes'hello there''escaped \'single quote\'''the path is C:\\\'some folder\'\\''1,2,3''{a:10} ${xyz}''"double quoted string"'It may be necessary to use multiple pairs of quotes to prevent yourshell from consuming quotation marks before they are passed to hydra.
$ python my_app.py '+foo="{a: 10}"'foo: '{a: 10}'$ python my_app.py '+foo={a: 10}'foo: a: 10Here are some best practices around quoting in CLI overrides:
Quote the whole key=value pair with single quotes, as in the first twoexamples above. These quotes are for the benefit of the shell.Do not quote keys.Only quote values if they contain a space. It will work if you always quotevalues, but it will turn numbers/dicts/lists into strings (as in the firstexample above).When you are quoting values, use double quotes to avoid collision with theouter single quoted consumed by the shell.Whitespaces in unquoted valuesUnquoted Override values can contain non leading or trailing whitespaces.For example, msg=hello world is a legal override (key is msg and value is the string hello world).Normally, your shell will interpret values with whitespaces as being multiple parameters (key=a b would be interpreted as key=a and b).To prevent this you can quote them with a single quote. For example:
$ python my_app.py 'msg=hello world'Note that trailing and leading whitespace are ignored, the above is equivalent to:
$ python my_app.py 'msg=hello world'Escaped characters in unquoted valuesHydra's parser considers some characters to be illegal in unquoted strings.These otherwise special characters may be included in unquoted values by escaping them with a \.These characters are: \()[]{}:=, \t (the last two ones being the whitespace and tab characters).
As an example, in the following dir is set to the string job{a=1,b=2,c=3}:
$ python my_app.py 'dir=job\{a\=1\,b\=2\,c\=3\}'As an alternative to escaping special characters with a backslash, the value containing the special character may be quoted:
$ python my_app.py 'dir=A[B'# parser error$ python my_app.py 'dir="A[B"' # ok$ python my_app.py 'dir=A\[B'# okPrimitivesid : oompa10, loompa_12null: nullint: 10, -20, 0, 1_000_000.float: 3.14, -10e6, inf, -inf, nan.bool: true, falsedot_path: foo.barinterpolation: ${foo.bar}, ${oc.env:USER,me}Constants (null, true, false, inf, nan) are case-insensitive.
importantAlways single-quote interpolations in the shell, to prevent replacement with shell variables:
$ python my_app.py 'dir=/root/${name}'In addition, more complex interpolations containing special characters may require being passed within a quoted value(note the extra double quotes surrounding the interpolation):
$ python my_app.py 'dir="${get_dir: {root: /root, name: ${name}}}"'Dictionaries and ListsListsfoo=[1,2,3]nested=[a,[b,[c]]]Dictionariesfoo={a:10,b:20}nested={a:10,b:{c:30,d:40}}Dictionaries are merged, not assigned. The following example illustrates the point:
Input configdb: driver: mysql user: ??? pass: ???db={user:root,pass:1234}db: driver: mysql user: root pass: 1234importantAlways single-quote overrides that contains dicts and lists in the shell.
Sweeper syntaxA choice sweep is comma separated list with two or more elements:
key=a,b# Simple sweep: ChoiceSweep(a, b)key="a,b","c,d"# Elements can be quoted strings, ChoiceSweep("a,b", "c,d")key=[a,b],[c,d]# Elements can be real lists, ChoiceSweep([a,b], [c,d])key={a:10, b:20},{c:30,d:40} # And dictionaries: ChoiceSweep({a:10, b:20}, {c:30,d:40})More sweeping options are described in the Extended Grammar page.
importantYou may need to quote your choice sweep in the shell.
FunctionsHydra supports several functions in the command line.See the Extended Grammar page for more information.
Working with your shellAll shells interprets command line inputs and may change what is passed to the process.A good way to determine what the shell is doing to your command is to echo it.
# bash output$ echo foo_{a:10,b:20} ${HOME} [b,c]*foo_a:10 foo_b:20 /home/omry build_helpers$ echo 'foo_{a:10,b:20}' '${HOME}' '[b,c]*'foo_{a:10,b:20} ${HOME} [b,c]*If in doubt, quote a command line element with a single quote (').
If you want to pass quotes to Hydra in a shell quoted string, it's best to pass double quotes.
$ echo '"hello world"'"hello world"You can use some shell specific commands to change their behavior, but the cost will be that their behavior will change.
BashYou can disable braces expansion, filename generation (globing) and hist expansion. Please note that this will changeyour shell behavior for the current session.
$ set +o braceexpand -o noglob +o histexpand$ echo key1={a:10,b:20} key2=${HOME} key=[b]*key1={a:10,b:20} key2=/home/omry key=[b]*# does not help with () though:$ echo key=choice(a,b,c)bash: syntax error near unexpected token '('$ echo 'key=choice(a,b,c)'key=choice(a,b,c)Other shellsSend a PR to add information about your favorite shell here.