GitLab

With scripting enabled, the main window now has an additional scripting widget that allows to interact with a Lua VM. The current proof-of-concept implementation creates a Lua VM for every instantiated CAMPVis pipeline while the console just interacts with the first pipeline's one.
However, the Lua VM concept has to be reiterated later anyway. It may make more sense to have just a single central Lua VM for the entire application, which is shared by each pipeline.

Fixed timing issue in CampVisPainter that led CampVisPainter::paint() being called before CampVisPainter::init(). CampVisPainter::paint() now ensures the class (and in particular the shader) to be correctly initialized.

fixes #564

Initial implementation of a Lua scripting layer

Over the last several months a scripting layer that allows pipelines to be defined in Lua has been developed. It uses SWIG to generate Lua modules with bindings for CAMPVis classes. It is an opt-in feature, and tries to be as non-intrusive to standard CAMPVis code as possible.

The implementation of the scripting layer has reached a state where it's possible to write fully-functional pipelines in Lua. In fact, 2 existing pipelines have been reimplemented in Lua for testing purposes and added to the project. In my opinion, this marks a good point to merge the initial implementation into the development branch — that would make it easier to test and improve it.

Naturally, there are still many rough edges that should eventually be dealt with, but they can be addressed separately as new features:
- bindings coverage is rather low
- Lua pipelines currently need to be statically registered
- pipeline definition syntax could be streamlined (e.g. by getting rid of the `instance` global variable)

The CMake-enabled version of Lua that CAMPVis' scripting layer uses
hasn't been updated in a while. Synchronise it with upstream now that
we're getting ready to merge the `swig` branch in.

References #1

`scripting/testapp.cpp` contains a simple console application that was
used to test CAMPVis' scripting layer before it was ready for
integration with the main GUI application. Now that Lua pipelines can be
registered and executed the same way regular ones are, the test
application is no longer needed. Consequently, this commit removes it
from the project.

References #1

Glue classes that live in the `scripting/glue` directory were mostly
undocumented, and the documentation of LuaPipeline was incomplete.

Improve the documentation of LuaPipeline and LuaVmState. Document
LuaTable, RegularLuaTable and GlobalLuaTable from the `scripting/glue`
directory.

References #1

Starting with commit d6fec679, pipelines need to provide their own light
source to function properly. Lua pipelines didn't, which caused them to
fail to render.

Add a SWIG wrapper for LightSourceProvider to make it accessible from
Lua. Instantiate and attach a light source in existing Lua pipelines.

References #1

Commit 293d43dd introduced breaking changes to the way invalidation
levels are handled. As a result, property and processor-related bindings
failed to compile.

Update all SWIG bindings affected by the above problem to make them
compatible with the current property and processor-related APIs.

References #1

LuaDist, the CMake-enabled distribution of Lua that the scripting
feature uses on Windows, provides several targets (lua, luac, wlua) for
which there's not much use in CAMPVis. This commit hides them from the
target list to make project files generated by CMake cleaner.

References #1

If CAMPVIS_BUILD_LIB_LUA was set but the Lua Git submodule wasn't checked
out, the user would get a generic "could not find CMakeLists.txt" error
message which wasn't of much help.

Log a custom error message that provides a hint how to fix the problem in
the above situation.

References #1

The FindLua CMake script would previously set LUA_FOUND if the directory
containing the source code of Lua existed without checking if it was
actually part of the build. This could lead to compilation errors if
CAMPVIS_BUILD_LIB_LUA was disabled.

Add an additional check to FindLua to only set LUA_FOUND if
CAMPVIS_BUILD_LIB_LUA is ON and the directory containing the source code
of Lua exists.

References #1

Scripting was so far enabled by default on the `swig` branch to make its
development easier. However, it should be an opt-in feature once it's
merged in. As that time is quickly approaching, the scripting feature has
been disabled by default.

References #1

Early in the development of the scripting feature the
`campvis-scripting-test` target was used to test bindings generated by
SWIG. Now that pipelines defined in Lua have been integrated into the
main application (where the bindings they use can be tested much more
thoroughly) the console test application is no longer needed and has
been removed.

References #1

SWIG wrappers for siglot only supported unary signals until now. This
commit adds support for all remaining signal arities to make it possible
to connect to arbitrary signals from Lua.

The implementation makes heavy use of templates but, due to the
limitations of VS 2010 (no variadic templates), still contains lots of
duplicated boilerplate code.

`sigslot.i` has been moved from `scripting/` to `ext/sigslot/` to keep
it close to the code it's wrapping.

References #1

Up until now it wasn't possible to manually disconnect slots defined in
Lua: they could only be disconnected automatically when their
corresponding signals were destroyed.

Add a special `disconnect` method to sigslot's Lua wrapper that makes it
possible to manually disconnect slots defined in Lua.

References #1

Now that support for signals and slots has been added to Lua pipelines,
each Lua state has to be protected with a mutex because it's not
thread-safe and different threads (running LuaPipeline or a processor
attached to it) may try to access it simultaneously.

The mutex needs to be recursive due to the fact that Lua code can
trigger the emission (or copying) of signals that have slots defined it
Lua connected to them. This in turn causes the state to be accessed from
a thread that, unbeknownst to it, already holds a lock on the mutex.

A pointer to the mutex is stored in the state's registry; this way code
that accesses Lua state directly (e.g. connections between sigslot's
signals and slots defined in Lua) can retrieve and lock it when
necessary.

References #1

Until now, Lua connections had only stub implementations of several
sigslot's API methods, including clone() and getdest(). This led to
crashes when signals that contained Lua connections were copied.

Provide proper (if somewhat forced) implementations of clone() and
getdest() for Lua connections. This makes Lua connections work with
signals that may be copied.

References #1

Now that BUILD_SHARED_LIBS is a top-level CAMPVis option, we could take
advantage of that and make the way we build Lua more robust. Also,
campvis-scripting has been marked as static library to prevent the build
from failing when BUILD_SHARED_LIBS is set.

References #1

To properly test the initial support for slot functions defined in Lua,
VolumeRendererDemo has been reimplement as a Lua pipeline.

References #1

Certain types of pipelines need to react to changes in their constituent
processors. CAMPVis uses a signal-slot mechanism (based on the sigslot
library) for this purpose. Up until now, however, there was no way to
connect to a signal from a pipeline defined in Lua.

This commit adds initial support for connecting Lua functions to
sigslot's signals. The current implementation only works with unary
signals, and should be considered a proof of concept. To test it,
AbstractProcessor's s_validated signal has been wrapped using SWIG.

References #1

Lua pipelines need to access certain protected properties of
AbstractPipeline but can't reference them directly as they don't
actually inherit from any class. ExtendedAutoEvaluationPipeline was
added to generated bindings to expose public getters for the properties
in question. LuaPipeline instances could then be cast to
ExtendedAutoEvaluationPipeline to give Lua code access to these new
methods.

However, now that HasPropertyCollection has been wrapped, the protected
properties can be accessed via its getProperty method. This approach is
preferred as it's much cleaner and simplifies our interface files. As a
result, ExtendedAutoEvaluationPipeline has been removed.

References #1

In order to reimplement VolumeRendererDemo in Lua
campvis::VolumeRenderer from the vis module has been wrapped using SWIG.

References #1

Several classes from campvis-core that are needed to reimplement
VolumeRendererDemo in Lua have been wrapped using SWIG. Among the
classes are: ImageData, CameraProperty,
TrackballNavigationEventListener, and their respective super-classes.

References #1

As a prerequisite to wrapping campvis::CameraProperty, bindings for two
new classes from TGT have been added: tgt::Vector3 and tgt::Camera.

References #1

As there's currently only one Lua pipeline, we could so far get away
with storing it in the `scripting` directory, giving it a generic name
and registering it manually. However, this approach won't work anymore
once additional Lua pipelines start to be implemented.

To support multiple Lua pipelines, implement a registration mechanism
for scripted pipelines based on PipelineFactory and similar to
PipelineRegistrar. It scans each active module's `pipelines` directory
for Lua pipelines, parses them and generates a registration header that,
when included, registers them with PipelineFactory.

As a result of the above, the test Lua pipeline had to be moved to
`modules/preprocessing/pipelines/` and could be renamed
ResamplingDemoLua.

References #1

To make it possible to access GlImageResampler's p_resampleScale
property from Lua, FloatProperty and the classes it depends on have been
wrapped using SWIG.

References #1

Now that TGT and campvis-core can be built as shared libraries, the few
remaining issues with the first Lua pipeline could be tracked and fixed.
As a result, the pipeline can be executed without any issues, and its
functionality exactly corresponds to that of ResamplingDemo. To achieve
this, several additional classes, methods and constants had to be
wrapped.

References #1

LuaPipeline used to create a Lua VM itself and interact with it
directly. That caused its code to be unnecassarily bloated and complex as
it dealt with rather low-level details.

Delegate the management of Lua VM's state to the recently introduced
LuaVmState class and use the new table helpers to simplify LuaPipeline.

References #1

Interacting with Lua using its low-level C API quickly becomes cumbersome
and error-prone. In such cases introducing an object-oriented glue layer
makes accessing Lua state considerably easier.

This commit adds several helper classes to campvis-scripting. The abstract
LuaTable and its concrete sub-classes RegularLuaTable and GlobalLuaTable
should make working with Lua tables more pleasant by reducing common
operations to a single method call. LuaVmState's task is to simplify the
management of Lua VM's state with various high level methods that do
automatic error checking.

The new classes are largely undocumented. This will be fixed in a future
commit.

References #1

There's a bug in SWIG_ADD_MODULE that breaks out-of-tree builds by not
creating wrapper output directories:
http://www.cmake.org/Bug/view.php?id=14459

Manually create missing directories for the time being to fix the
problem.

References #1

All interface files used to live under `scripting/`. However, that
separated them from the code they were binding and clustered them
unnecessarily.

Make each interface file part of the CAMPVis component whose code it's
binding. This fixes the issues mentioned above and makes it possible to
build Lua modules conditionally, depending on what components of CAMPVis
are enabled.

References #1

When Lua is included in the build, its CMakeLists.txt enables
BUILD_SHARED_LIBS if it isn't explicitly disabled. This breaks the build
as CMake tries and fails to build all of CAMPVis' libraries as shared
libraries.

Disable BUILD_SHARED_LIBS before including `ext/lua` to prevent that.
Please note that this is a temporary fix; we'd like to eventually build
Lua and at least some of CAMPVis' libraries as shared libraries.

References #1

There were several problems with how CAMPVis' Lua modules were built:
- modules were placed in the same directory as all other CAMPVis'
  libraries, which polluted Lua's module path and could lead to name
  clashes
- SWIG interface files included instead of importing one another; this
  resulted in duplicated bindings being generated

To fix these probles place all of CAMPVis' Lua modules in one
sub-directory and let the code know what that sub-directory is so that
it can instruct Lua to search it. Moreover, use %import instead of
%include when one SWIG interface file needs to have access to
definitions from another one.

References #1

This commit marks the start of work on making Lua pipelines as easy to
register and instantiate as regular ones. A sample Lua pipeline has been
made available to the application so that problems with integrating Lua
pipelines can be discovered and fixed.

References #1

To check if existing bindings for property classes work, the test Lua
pipeline has been updated to modify processor properties in its init
method.

References #1

To make it possible to interact with TransferFunctionProperties from
Lua, several TransferFunction-related classes have been wrapped.

References #1

The header with SWIG run-time functions was generated in the PRE_BUILD
phase which is only fully supported by Visual Studio builds. Add an
explicit dependency on swigluarun.h to the campvis-scripting target to
make sure that the former exists before the latter is built.

References #1

TGT-related glue code has been moved to a new interface file: tgt.i. It
wraps and instantiates tgt::Vector2 and (starting with this commit)
tgt::Vector4.

Many methods of those classes and a lot of typedefs haven't been wrapped
yet. This will be fixed when necessary in future commits.

References #1

Out-of-tree builds were broken due to Lua and Swig header files not
being correcly included. The state of Visual Studio builds was even
worse: they failed because Lua modules, scripts and libraries were
misplaced.

The issues mentioned above have been fixed by making the build system
properly deal with out-of-tree builds and multi-config generators.

References #1

Lua functions invoked by LuaPipeline had no access to the pipeline
instance and, as a result, couldn't configure or modify it in any way.

In order to make it possible to actually define pipelines in Lua, each
script can now access the C++ pipeline object it's supposed to augment
via a global variable named instance. This variable is injected into the
script by LuaPipeline and can be used to call standard pipeline methods
(e.g. addProcessor).

The functionality described has been tested by implementing a pipeline
constructor that creates and registers several processors.

References #1