Configuration File With List Of Key-value Pairs In Python

I have a python script that analyzes a set of error messages and checks for each message if it matches a certain pattern (regular expression) in order to group these messages. For

Solution 1:

I sometimes just write a python module (i.e. file) called config.py or something with following contents:

config = {
    'name': 'hello',
    'see?': 'world'
}

this can then be 'read' like so:

from config import configconfig['name']
config['see?']

easy.

Solution 2:

You have two decent options:

1. Python standard config file format using ConfigParser
2. YAML using a library like PyYAML

The standard Python configuration files look like INI files with [sections] and key : value or key = value pairs. The advantages to this format are:

• No third-party libraries necessary
• Simple, familiar file format.

YAML is different in that it is designed to be a human friendly data serialization format rather than specifically designed for configuration. It is very readable and gives you a couple different ways to represent the same data. For your problem, you could create a YAML file that looks like this:

file .* does not exist : file not found
user .*not found : authorization error

Or like this:

{ file.*does not exist:filenotfound,
  user.*not found:authorizationerror }

Using PyYAML couldn't be simpler:

importyamlerrors= yaml.load(open('my.yaml'))

At this point errors is a Python dictionary with the expected format. YAML is capable of representing more than dictionaries: if you prefer a list of pairs, use this format:

-
  - file .* does not exist 
  - file not found
-
  - user .* not found
  - authorization error

Or

[ [file .* does not exist, file not found],
  [user .* not found, authorization error]]

Which will produce a list of lists when yaml.load is called.

One advantage of YAML is that you could use it to export your existing, hard-coded data out to a file to create the initial version, rather than cut/paste plus a bunch of find/replace to get the data into the right format.

The YAML format will take a little more time to get familiar with, but using PyYAML is even simpler than using ConfigParser with the advantage is that you have more options regarding how your data is represented using YAML.

Either one sounds like it will fit your current needs, ConfigParser will be easier to start with while YAML gives you more flexibilty in the future, if your needs expand.

Best of luck!

Solution 3:

I've heard that ConfigObj is easier to work with than ConfigParser. It is used by a lot of big projects, IPython, Trac, Turbogears, etc...

From their introduction:

ConfigObj is a simple but powerful config file reader and writer: an ini file round tripper. Its main feature is that it is very easy to use, with a straightforward programmer's interface and a simple syntax for config files. It has lots of other features though :

• Nested sections (subsections), to any level
• List values
• Multiple line values
• String interpolation (substitution)
• Integrated with a powerful validation system
  • including automatic type checking/conversion
  • repeated sections
  • and allowing default values
• When writing out config files, ConfigObj preserves all comments and the order of members and sections
• Many useful methods and options for working with configuration files (like the 'reload' method)
• Full Unicode support

Solution 4:

I think you want the ConfigParser module in the standard library. It reads and writes INI style files. The examples and documentation in the standard documentation I've linked to are very comprehensive.

Solution 5:

If you are the only one that has access to the configuration file, you can use a simple, low-level solution. Keep the "dictionary" in a text file as a list of tuples (regexp, message) exactly as if it was a python expression:

[
("file .* does not exist", "file not found"),
("user .* not authorized", "authorization error")
]

In your code, load it, then eval it, and compile the regexps in the result:

f = open("messages.py")
messages = eval(f.read()) # caution: you must be sure of what's in that file
f.close()
messages = [(re.compile(r), m) for (r,m) in messages]

and you end up with a list of tuples (compiled_regexp, message).