Pylux, Mac OS X, non-blender python

[mtoivo]

Hi.

I've been trying to compile pylux against the python libs installed on my machine (OS X 10.6), not the blender-bundled one. The reason is Luxfire (manager for distributed rendering), and for that to work the pylux module has to work with the python installed. I have specified the PYTHON_INCLUDE_PATH and PYTHON_LIBRARY in CMake to point to the correct paths (trying to get 3.1 or 3.2 working, no idea if 2.6 works). After wondering why it still doesn't work, I tried the command otool -L pylux.so and got the following output:

Code: Select all

pylux.so:
   /System/Library/Frameworks/OpenCL.framework/Versions/A/OpenCL (compatibility version 1.0.0, current version 1.0.0)
   /System/Library/Frameworks/AGL.framework/Versions/A/AGL (compatibility version 1.0.0, current version 1.0.0)
   /System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL (compatibility version 1.0.0, current version 1.0.0)
   /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.3)
   /usr/lib/libbz2.1.0.dylib (compatibility version 1.0.0, current version 1.0.5)
   /System/Library/Frameworks/Python.framework/Versions/2.6/Python (compatibility version 2.6.0, current version 2.6.1)
   /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.9.0)
   /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 123.0.0)


Note the line /System/Library/Frameworks/Python.framework/Versions/2.6/Python which clearly points, that the pylux has indeed been compiled against wrong python. To my understanding, at least. Any idea why that happens? Or is that even the reality? And where the heck it gets that path? I figured, that maybe python-config is to blame and removed any traces from version 2.6 of python-config, with the same exact results. Before even getting to possible issues with the Luxfire it self, I tried running python pyluxconsole.py, but it just gives out error Symbol not found: _luxGetAttributeType. Any help would be very much appreciated!

Mikko

[J the Ninja]

/System/Library/Frameworks/Python.framework/Versions/2.6/Python is the Apple-provided python all 10.6 installs come with. I'm assuming that isn't what you really wanted, you wanted a user-installed version of python 3? Lines 912-927 of cmake lists seem to deal with this:

Code: Select all

# SET(CMAKE_USE_PYTHON_VERSION 3.2)

IF(APPLE)
   IF(OSX_OPTION_PYLUX)
      # use Blender python libs for static compiling !
      SET(PYTHON_LIBRARIES ${CMAKE_SOURCE_DIR}/../macos/lib/BF_pythonlibs/py32_uni_intel/libbf_python_ext.a ${CMAKE_SOURCE_DIR}/../macos/lib/BF_pythonlibs/py32_uni_intel/libbf_python.a)
      SET(PYTHON_INCLUDE_PATH ${CMAKE_SOURCE_DIR}/../macos/include/Python3.2)
      SET(PYTHONLIBS_FOUND ON)
   ELSE(OSX_OPTION_PYLUX)
      # compile pylux for genral purpose against Python framework
      FIND_LIBRARY(PYTHON_LIBRARY Python )
      FIND_PATH(PYTHON_INCLUDE_PATH python.h )
      MARK_AS_ADVANCED (PYTHON_LIBRARY)
      SET(PYTHONLIBS_FOUND on)
      SET(PYTHON_LIBRARIES ${PYTHON_LIBRARY})
   ENDIF(OSX_OPTION_PYLUX)


It seems you need to disable the OSX_OPTION_PYLUX checkbox, but somehow set your default python as the 3.* install and not the Apple 2.6 one? (I'm officially lost at this point now)

[mtoivo]

...2.6/Python is the Apple-provided python all 10.6 installs come with. I'm assuming that isn't what you really wanted, you wanted a user-installed version of python 3?

Correct.

It seems you need to disable the OSX_OPTION_PYLUX checkbox

That also is correct. I have disabled it.

, but somehow set your default python as the 3.* install and not the Apple 2.6 one? (I'm officially lost at this point now)

Yep. PYTHON_INCLUDE_PATH is set to /System/Library/Frameworks/Python.framework/Headers and PYTHON_LIBRARY is set to /System/Library/Frameworks/Python.framework. I have 3 versions installed there, Headers is symlink to Versions/Current/Headers, where "Current" is symlinked to 3.2 (a change made by me). Still, what ever I do, somehere the freaking 2.6 version gets lured into the pylux build, and I have no idea why.

[J the Ninja]

...2.6/Python is the Apple-provided python all 10.6 installs come with. I'm assuming that isn't what you really wanted, you wanted a user-installed version of python 3?

Correct.

It seems you need to disable the OSX_OPTION_PYLUX checkbox

That also is correct. I have disabled it.

, but somehow set your default python as the 3.* install and not the Apple 2.6 one? (I'm officially lost at this point now)

Yep. PYTHON_INCLUDE_PATH is set to /System/Library/Frameworks/Python.framework/Headers and PYTHON_LIBRARY is set to /System/Library/Frameworks/Python.framework. I have 3 versions installed there, Headers is symlink to Versions/Current/Headers, where "Current" is symlinked to 3.2 (a change made by me). Still, what ever I do, somehere the freaking 2.6 version gets lured into the pylux build, and I have no idea why.

NEVER modify /System. It's for stuff that ships with the OS only. The entire reason the /Library directory exists is so that you can add frameworks, kexts, fonts, and whatever else system-wide while leaving /System alone. You should remove all of that and install python 3 to /Library/Frameworks, then set the include path to there.

[mtoivo]

NEVER modify /System. It's for stuff that ships with the OS only. The entire reason the /Library directory exists is so that you can add frameworks, kexts, fonts, and whatever else system-wide while leaving /System alone. You should remove all of that and install python 3 to /Library/Frameworks, then set the include path to there.

I figured I shouldn't, and planned to restore it to default 10.6 configuration, which I now did. The only reason I tweaked it was because I couldn't get rid of the 2.6 -thing. I now defined the path to /Library/Frameworks/Python.framework, it still gets linked agains 2.6...

[dougal2]

Can you check that your boost libs are built against the correct python lib? It's the linkage of python -> boost -> pylux which will ultimately determine which version of python the pylux module will work with.

[mtoivo]

Can you check that your boost libs are built against the correct python lib? It's the linkage of python -> boost -> pylux which will ultimately determine which version of python the pylux module will work with.

Didn't find any sure method for that, but dir "py32_uni_intel" (found in sources macos/lib/BF_pythonlibs) gives a hint it's 3.2. So I think the 2.6 version tag doesn't come from there. I could try to compile the libboost too.

[mtoivo]

My mistake, the BF_pythonlibs is of course Blender Foundation python, and it is not used when OSX_OPTION_PYLUX is not set. Any help digging the python version from libboost?

[jensverwiebe]

The boost in macos repo is build against python 3 !
Atm i hadn´t looked into what luxfire needs, but you can set the findpath for python by hand in the cmake gui.

I know it may be inconvenient that a special pylux is needed to work with Blender. I only found this way linking against
BF_PYTHON components to avoid mismatch errors at runtime. ( blender uses python statically !! )

So if you need a pylux for general use, disable OSX_OPTION_PYLUX and make sure python 3.2 framework is used for linking.
This may be only succesful when doing XCode builds, due i found cmake often muddling the pathes. But XCode build is our choice anyway.
You may try to watch the includes and lib for python in XCode directly if they where properly translated ny cmake generation.
I already build such pylux in former times ( not recently ) so this should work.

On the other hand the statically linked pylux should work standalone as well, but i never tested it deeply.

EDIT: found it: you must change the search order in framework searchpath of pylux to have /Developer/SDKs/MacOSX10.6.sdk/Library/Frameworks first !
I must lookup if i can get cmake to do this for us by default later.

Must look like this after:

Code: Select all

/Volumes/Daten1TB/LuxRender_devel/lux_mainbranch_restructure_test_dynpylux/Release/pylux.so:
   /System/Library/Frameworks/OpenCL.framework/Versions/A/OpenCL (compatibility version 1.0.0, current version 1.0.0)
   /System/Library/Frameworks/AGL.framework/Versions/A/AGL (compatibility version 1.0.0, current version 1.0.0)
   /System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL (compatibility version 1.0.0, current version 1.0.0)
   /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.3)
   /usr/lib/libbz2.1.0.dylib (compatibility version 1.0.0, current version 1.0.5)
   /Library/Frameworks/Python.framework/Versions/3.2/Python (compatibility version 3.2.0, current version 3.2.0)
   /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.9.0)
   /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.0)


Code: Select all

jens-macpro:Release jensverwiebe$ file pylux.so
pylux.so: Mach-O universal binary with 2 architectures
pylux.so (for architecture i386):   Mach-O bundle i386
pylux.so (for architecture x86_64):   Mach-O 64-bit bundle x86_64


I tested it initially and it works:

Code: Select all

jens-macpro:Release jensverwiebe$ python3.2Python 3.2 (r32:88445, Mar  1 2011, 20:41:46)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pylux
>>>


Hope this helps. Still problems? Lemme know.....

Jens

[mtoivo]

Hi Jens, thanks for the reply.

Can you tell me where exactly I can change the framework searchpath in XCode? I can't seem to find the place myself.