Building on Debian - LuxRender Wiki
Luxrender GPL Physically Based Renderer

Building on Debian

Personal tools

From LuxRender Wiki

Jump to: navigation, search

This tutorial provides a walk-through of building Lux and Luxrays on Debian systems. The tutorial proposes a directory structure for LuxRender and 3rd Party installations, and provides a dependency matrix to help readers install required libraries quickly.

A general approach of this tutorial is to try to isolate LuxRender dependencies from OS libraries, e.g. having “most” 3rd Party packages and libraries needed by LuxRender on a separate folder. This way, we can test different libraries quickly, for example Boost 1_54_0 and Boost 1_55_0.

This tutorial builds a debug version of Lux and Luxrays, however it should be straightforward to build release as well.

Contents

Detailed Host System Configuration

Item Version/Description
Architecture/CPU i686/32Bit
OS Debian Wheezy
Python 3.2.2
Boost 1_55_0
CMake 2.8.9
GCC 4.8.2
Make 3.81
Luxrays 2180
Lux 4637
OpenCL NOT Tested

Summary of Libraries Dependencies

Tip You can install all the libraries below by typing:

$ sudo apt-get install <library_name>

Library Python 3.3.2 Boost 1_55_0 Luxrays Lux Other Comment
libfreeimage-dev x
libgdbm-dev x
tk-dev x
libbz2-dev x
libglew-dev x
freeglut3-dev x
libfftw3-dev x
bison x
flex x
libopenexr-dev x
qt-sdk x Not the entire SDK is needed, could be stripped down
libpng12-dev x
doxygen x Optional
tar x Archiving Utility
p7zip-full x Compression Utility
gcc x GNU C Compilers
g++ x GNU C++ Compilers
make x Make Utility
cmake x Build System
mercurial x Revision Control System
binutils x Binary utilities needed for creating binaries
xdot x To view dependency graphs created by CMake

Suggested Directory Structure for Development

~/workspace

→ apps (3d related apps, e.g. blender, makehuman, etc. No compilation needed)
→ __install__ (the installation files themselves)
→ eclipse.7z
→ blender.7z
→ makehuman.deb
→ eclipse (eclipse is installed here)
→ blender
→ assets (all graphical assets like textures, soundtracks, etc.)
→ texture
→ scenes
→ models
→ dev (all source code related development )
→ stage (3rdParty libraries that need compilation )
→ __install__
→ python
→ python-3.3.2.7z (downloaded from the Internet)
→ 3.2.2 (extract version 3.3.2 here)
→ boost
→ boost_1.53.0.7z (downloaded from the Internet)
→ 1_55_0 (extract version 1_55_0 here)
→ 1_54_0
→ boost (after compilation, boots gets installed here)
→ 1_55_0
→ 1_54_0
→ python (after compilation, python gets installed here)
→ 3.3.2
→ 2.7.6
→ src (all sources are here…)
→ navelgaze (personal project + keep other projects at this level, e.g. blender)
→ lux
→ prep_lux_debug.sh (helper shell script to simplify cmake invocation)
→ prep_lux_release.sh (TODO: merge all these files in one)
→ prep_luxrays_debug.sh
→ prep_luxrays_release.sh
→ lux
→ luxrays
→ luxblend25
→ build (needed for out of source build)
→ lux
→dependency (dependency graphs, use xdot to visualize)
→debug
→release
→ luxrays
→dependency
→debug
→release
→ projects (e.g. IDE workspace)
→ lux (default project used to test/debug lux…)

Building and Installation Directions

Python

Install Dependencies

$sudo apt-get install libfreeimage-dev (TODO: this should go to Luxrays, double check if it’s needed here)
$sudo apt-get install libgdbm-dev
$sudo apt-get install tk-dev

Download

Download Python 3.3.2 from Python website. Note as of writing this document, a newer version of Python already is released.

Tip To follow the rest of the instructions below as is (i.e. without changing files paths), you need to store the downloaded library in the directory indicated in the Suggested Directory Structure

Uncompress

~/workspace/dev/stage/__install__/$python/7z x Python-3.3.2.tar.xz
~/workspace/dev/stage/__install__/$python/tar xvf Python-3.3.2.tar (f=file, v=verbose, x=extract)
~/workspace/dev/stage/__install__/$python/mv Python-3.3.2 3.3.2 (to follow our directory structure, change names)
~/workspace/dev/stage/__install__/$python/rm Python-3.3.2.tar (not needed any more)

Build and Install

~/workspace/dev/stage/$ mkdir python python/3.3.2 (TODO: do we need to create folders manually or will python create them automatically?)
~/workspace/dev/stage/__install__/python/3.3.2/$ ./configure --prefix=$HOME/workspace/dev/stage/python/3.3.2 --enable-shared ( to create shared python library, as opposed to static library .a only)
~/workspace/dev/stage/__install__/python/3.3.2/$ make -j4 (or -l0.5 → max load cpu 50%)
~/workspace/dev/stage/__install__/python/3.3.2/$ make test (unit tests…)
~/workspace/dev/stage/__install__/python/3.3.2/$ make install

Clean

~/workspace/dev/stage/__install__/python/3.3.2/$ make clean

Boost

Introduction to Boost

Most Boost libraries are headers files only (.hpp), so they don’t require compilation. However, some libraries must be compiled in advance, for example : FileSystem, GraphParallel and IOStreams. Please check the Boost documentation for the full list. Note that Libraries that don't need compilation are called Header­Only Libraries, whereas Libraries that need compilation are called Binary Libraries.

Tips

./bootstrap.sh ­­--help → is your main tool to build binary libraries, use --help to get more info about this tool
./bootstrap.sh ­­--show-libraries → display a list of Boost libraries that require build

For further information on boost installation, refer to this PDF

Install Dependencies

$ sudo apt-get install libbz2-dev

Download

Download from Boost website

Uncompress

~/workspace/dev/stage/__install__/boost$ 7z x boost_1_55_0.7z (extract to current dir)
~/workspace/dev/stage/__install__/boost$ tar xvf boost_1_55_0.tar
~/workspace/dev/stage/__install__/boost$ mv boost_1_55_0 1_55_0
~/workspace/dev/stage/__install__/boost$ rm boost_1_55_0.tar (not needed any more)

Install Patches

This is a reminder that you should check for Boost Patches available on the Boost website. Use the patch command to install them. As of writing this tutorial there is one patch for Boost version 1_55_0.

Build and Install

~/workspace/dev/stage/mkdir boost boost/1_55_0 (TODO: do we need this, or will Boost create it)
~/workspace/dev/stage/__install__/boost/1_55_0$ ./bootstrap.sh --with-libraries=all --with-python-root=$HOME/workspace/dev/stage/python/3.3.2/bin --with-python-version=3.3 --prefix=$HOME/workspace/dev/stage/boost/1_55_0 (i.e. point to python installation dir and where to install boost)
~/workspace/dev/stage/__install__/boost/1_55_0$ mkdir build_dir (it’s recommended to have out of source build)
~/workspace/dev/stage/__install__/boost/1_55_0$ ./b2 -j4 --build-dir=build_dir include=$HOME/workspace/dev/stage/python/3.3.2/include/python3.3m install | tee out.txt

Tip You can check out .txt for details of compilation.

Clean

./b2 clean (TODO: do we need to delete build_dir manually?)

Luxrays

Tip All lux projects are located @ http://src.luxrender.net/

Install Dependencies

$ sudo apt-get install libglew-dev
$ sudo apt-get install freeglut3-dev
$ sudo apt-get install openimageio-tools

Clone Source Code

~/workspace/dev/src/lux$ mkdir luxrays
~/workspace/dev/src/lux$ hg clone http://src.luxrender.net/luxrays luxrays

Create a Shell Script to Simplify Cmake Invocation

The purpose of this shell script to tell CMake where our libraries are (i.e. boost and python), whether to build Release or Debug, etc. Remember that we are trying to isolate OS libraries from Lux (to facilitate testing different libraries quickly), hence we need to provide CMake with the location of these libraries.

Use a text editor to create prep_luxrays_debug.sh in ~/workspace/dev/src/lux

prep_luxrays_debug.sh

---------------------
#!/bin/bash
# We should use BOOST_ROOT instead of BOOST_SEARCH_PATH but currently it's  overwritten by local lux cmake scripts
cmake --debug-output --trace --graphviz=/$HOME/workspace/dev/build/luxrays/dependency/luxrays \
-H$HOME/workspace/dev/src/lux/luxrays \
-B$HOME/workspace/dev/build/luxrays/debug \
-DCMAKE_BUILD_TYPE=DEBUG \
-DCMAKE_VERBOSE_MAKEFILE=TRUE \
-DLUXRAYS_DISABLE_OPENCL=ON \
-DBoost_NO_SYSTEM_PATHS=TRUE \
-DBOOST_SEARCH_PATH=$HOME/workspace/dev/stage/boost/1_55_0 \
-DPYTHON_LIBRARY=$HOME/workspace/dev/stage/python/3.3.2/lib/libpython3.3m.so \
-DPYTHON_INCLUDE_DIR=$HOME/workspace/dev/stage/python/3.3.2/include/python3.3m  
---------------------

~/workspace/dev/src/lux$ chmod +x prep_luxrays_debug.sh
~/workspace/dev/src/lux$ ./prep_luxrays_debug.sh (hopefully no errors!)

Tips

  1. Cmake options --debug-output and --trace produces a very lengthy output while running cmake and make. This helps in troubleshooting compilation/linking problems, however you can remove them if you don't prefer such output
  2. You can redirect output of prep_lux_debug.sh script to file by typing:
    ~/workspace/dev/src/lux$ ./prep_lux_debug.sh > prep_lux_debug_result.txt 2>&1

Workaround For a “Strange” Linker Error (FIXME)

Building Luxrays as is would produce linking problems related to missing python libraries in benchsimple, luxcore, and other samples in Luxrays. It needs to be investigated more, as I am not sure why the problem happens. To reproduce the problem, skip this section and jump to next section.

The workaround is appending ${PYTHON_LIBRARY} to target_link_libraries in the following four files:
luxrays/samples/benchsimple/CMakeLists.txt
luxrays/samples/luxcoredemo/CMakeLists.txt
luxrays/samples/luxcorescenedemo/CMakeLists.txt
luxrays/samples/smallluxgpu4/CMakeLists.txt

For example, benchsimple/CMakeLists.txt would look similar to this:

---------------------
include_directories(${LuxRays_INCLUDE_DIR})
link_directories (${LuxRays_LIB_DIR})

add_executable(benchsimple benchsimple.cpp)
target_link_libraries(benchsimple luxrays ${PYTHON_LIBRARY})
---------------------

Tip Remember to re-run the previous prep_luxrays_debug.sh script to regenerate the makefiles.

Build Using Make

~/workspace/dev/build/luxrays/debug$ make -j4 (or -l0.5 → max load cpu 50%)

Dependency Visualization

~/workspace/dev/build/luxrays/dependency$ xdot luxrays

Lux

Build Luxrays

This is a reminder that Lux depends on luxRays, so you must first build Luxrays

Install Dependencies

$ sudo apt-get install bison
$ sudo apt-get install flex
$ sudo apt-get install libopenexr-dev
$ sudo apt-get install qt-sdk (TODO: not the entire sdk is needed, could be stripped down)
$ sudo apt-get install libpng12-dev
$ sudo apt-get install doxygen (optional)

Clone Source Code

~/workspace/dev/src/lux$ mkdir lux
~/workspace/dev/src/lux$ hg clone http://src.luxrender.net/lux lux

Create a Shell Script to Simplify Cmake Invocation

Use a text editor to create prep_lux_debug.sh in ~/workspace/dev/src/lux

prep_lux_debug.sh

---------------
#!/bin/bash
cmake --debug-output --trace --graphviz=/$HOME/workspace/dev/build/lux/dependency/lux \
-H$HOME/workspace/dev/src/lux/lux \
-B$HOME/workspace/dev/build/lux/debug \
-DCMAKE_BUILD_TYPE=DEBUG \
-DCMAKE_VERBOSE_MAKEFILE=TRUE \
-DLUXRAYS_DISABLE_OPENCL=ON \
-DLuxRays_HOME=$HOME/workspace/dev/build/luxrays/debug \
-DBoost_NO_SYSTEM_PATHS=TRUE \
-DBOOST_ROOT=$HOME/workspace/dev/stage/boost/1_55_0 \
-DPYTHON_LIBRARY=$HOME/workspace/dev/stage/python/3.3.2/lib/libpython3.3m.so \
-DPYTHON_INCLUDE_DIR=$HOME/workspace/dev/stage/python/3.3.2/include/python3.3m 
---------------

~/workspace/dev/src/lux$ chmod +x pre_lux_debug.sh
~/workspace/dev/src/lux$ ./prep_lux_debug.sh

Build Using Make

~/workspace/dev/build/lux/debug$ make -l0.50 (or use -j#)

Dependency Visualization

~/workspace/dev/build/lux/dependency$ xdot lux

Luxblend25 + Blender Integration (IMPROVE)

~/workspace/dev/src/lux/hg clone http://src.luxrender.net/luxblend25 luxblend25
~/workspace/apps/blender-2.68.../2.68/scripts/addons$ ln -s $HOME/workspace/dev/src/lux/luxblend25/src/luxrender
~/export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/workspace/dev/stage/boost/1_55_0/lib
~/workspace/apps/blender-2.68.../$./blender &

Then in Blender, change system settings to load LuxRender.

Tips on Resolving Shared Libraries Dependencies

If you try to run a binary (for example luxconsole), and the OS complains about missing shared libraries (for example libboost_python.so), then you should do the following:

  1. Make sure the library exists in the first place (usually in /etc/lib, /etc/local/lib, $HOME/workspace/dev/stage/boost/...), and
  2. The location of the “missing” shared library should be defined to the dynamic loader, you can do this by either:
    1. permanently, edit /etc/ld.so.conf to include the path required for the shared library, or
    2. create a shell script the export the location of the shared library, then call your binary, e.g:
#!/bin/bash
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/required_lib
your_binary

More Notes

  1. $ ldd binary_name | grep “not found” →prints shared libraries used by a binary that are missing
  2. $ env → prints a list of environment variables
  3. $ echo $LD_LIBRARY_PATH prints the content of LD_LIBRARY_PATH environment variable
  4. You must export an environment variable in order to be seen by child processes, to do so type: $ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/required_lib
  5. To make sure that you have exported an environment variable, type : $ export -p → this will list all exported environment variables.

Todo

  1. Investigate why we need to add ${PYTHON_LIBRARY} in CMakeLists.txt for four luxrays targets: benchsimple, luxcore, luxcoredemo,...
  2. Test on 64 bit Machine
  3. Test on different Linux distributions such as Ubuntu
  4. Test OpenCL support
  5. Test Release
  6. Test Lux, Luxrays Installation, i.e. make install
  7. Create Unit Tests (would they be S/W related only, or also 3D, e.g. standard scenes)
  8. Use a continuous build server, e.g. Hudson.
  9. Combine different shell scripts, created by this tutorial into one parameterized one

Troubleshooting

  1. If you find cmake insists on not using the new values you assigned in your prep_lux_debug.sh, or similar files, then most probably it uses the cached version. To solve the problem, delete the cache file, i.e. CMakeCache.txt and try again.
  2. If you get nonsense compiler/linker errors, then it’s best to do a clean source code check out and clean build.