From LuxRender Wiki
At the present time, the LuxFire system is a work-in-progress. Installation and use shouldn't be attempted unless you have almost developer-level proficiency in software setup.
LuxFire is a multi-component distributed network rendering system for the LuxRender rendering engine. It is architectured such that some components can be run on different machines across the local network.
LuxFire aims to be compatible with both Python 2 and Python 3. The choice of which Python version to use depends on the LuxFire configuration you wish to use, since not all supporting libraries are yet available for Python 3.
Note that running components under different versions of Python on the same network is not supported.
$ hg clone http://src.luxrender.net/luxfire
$ hg clone http://bitbucket.org/luxrender/luxfire
Required Python libraries
LuxFire requires sqlalchemy and jinja2 to operate. For MySQL databases, you also need the mysqldb library.
For ubuntu based linux systems you can install the required python libraries with these commands:
$ sudo apt-get install python-sqlalchemy python-jinja2
$ sudo apt-get install python-mysqldb
$ sudo apt-get install python3-sqlalchemy python3-jinja2
In the following sections, for the supplied commands to work it is assumed that the current directory relative to where you issued the hg clone command is:
LuxFire comes with a copy of the Pyro networking library, which enables fancy local networking features. For the LuxFire system to operate, there must be exactly one Pyro name server running on the local network, with broadcasting enabled. Broadcasting is not enabled when the name server is running on the localhost address 127.0.0.1
$ python3 -m Pyro.naming -n 192.0.X.X
LuxFire comes with a stub package for LuxRender, which contains some utility functions used by the system. In order to use the Renderer component, you should put your binary pylux file inside the correct folder in this package. Note that the pylux binary must be built against the correct python version for your LuxFire installation.
- For python 2: add LuxRender/py2/pylux.[so|pyd]
- For python 3: add LuxRender/py3/pylux.[so|pyd]
The main LuxFire package contains several sub-components as described below. Some components expose modules which can be executed directly to perform functions for LuxFire or to start servers.
LuxFire uses a central database to hold data about users and jobs on the system. The interface to the database is abstracted via the sqlalchemy python library, so any database supported by sqlalchemy can be used by LuxFire. By default, LuxFire will use a local sqlite3 database file for storage.
So far, the following configurations have been tested:
- Python 2: sqlite3 (built-in), MySQL (requires python-mysqldb package)
- Python 3: sqlite3 (built-in)
The LuxFire.Database.Create command will initialise a new database installation and create a default luxfire.cfg configuration file:
$ python3 -m LuxFire.Database.Create --help Usage: LuxFire.Database.Create [options]
Creates the database required for LuxFire. If a non-default database connection string is specified, it will be saved into the luxfire.cfg file so that LuxFire can connect to it automatically the next time it is run Options: -h, --help show this help message and exit -d STR, --database=STR Database connection string to use. See sqlalchemy docs for supported connection types -v, --verbose Show database creation progress -b ADDR, --bind=ADDR Specify the local IP/hostname to bind services to. This would usually be the IP address of this machine's main network adaptor. It is not advisable to bind to IP addresses that are accessible on the internet.
When the database is created, it will also create a default administrative user, called root, with the password admin.
TODO: describe [LuxFire] section in luxfire.cfg
The Dispatcher is a render queue manager for the LuxFire system. It will read the following items from the luxfire.cfg file:
[Dispatcher] process_interval = 5 ; Number of seconds between processing the queue max_items_per_worker = 10 ; Max. number of items to process per interval
[LocalStorage] ; TODO
[NetworkStorage] ; TODO
The Dispatcher can be started as a server with the following command:
$ python3 -m LuxFire.Dispatcher.Server
You should (at the present time) run only ONE Dispatcher in the LuxFire system.
Any of the LuxFire component servers can be stopped with CTRL-C or a SIGINT.
To connect to an existing Dispatcher on the network from a python console, use the client command:
$ python3 -im LuxFire.Dispatcher.Client
The Renderer module is responsible for creating LuxRender rendering processes on the network. It is configurable in the luxfire.cfg file:
[Renderer] threads_per_server = 4 ; Number of rendering threads to use per server
Running this component as a server requires that a pylux module is put into the LuxRender package, and can be run using this command:
$ python3 -m LuxFire.Renderer.Server
You can run as many Renderer.Servers on the network as needed, you can even run two or more on the same machine (providing the threads_per_server setting is set correctly!).
To connect to existing Renderers on the network from a python console:
$ python3 -im LuxFire.Renderer.Client
The Web module provides a web-based administration and monitoring interface for the LuxFire system. It uses the following luxfire.cfg section:
[Web] port = 9080 ; Listening port for Web server
Running the server requires using the same database as the running Dispatcher; in the case of using sqlite, this means running on the same machine. These components can run on different machines if using a central MySQL server.
Running as a server:
$ python3 -m LuxFire.Web.Server [-v]
This command takes an options -v/--verbose option to print a request log to the console. With the default configuration, you can now access the interface via http://127.0.0.1:9080/ using username root and password admin.
Also supplied is a "standalone" script which will, by default, run one instance of each of the server components using one command:
$ ./luxfire_standalone.py --help Usage: luxfire_standalone.py [options]
All-in-one launcher script for a standalone render queue. By default, this script will start a Pyro name server, a LuxFire.Dispatcher.Server, LuxFire.Renderer.Server and a LuxFire.Web.Server. You can disable any of these with the options below. Options: -h, --help show this help message and exit -n, --no-nameserver Do not start a Pyro name server. -d, --no-dispatcher Do not start a LuxFire.Dispatcher.Server -r, --no-renderer Do not start a LuxFire.Renderer.Server -w, --no-webserver Do not start a LuxFire.Web.Server -v, --verbose Show more output while running
Note, that before running this command you should have run LuxFire.Database.Create to initialise the database and LuxFire configuration.