The functionality of RmlUi is described fully in the C++ Manual; this manual defines the Lua interface to the RmlUi objects described there. Not all aspects of RmlUi are accessible from Lua; for example, custom decorators can only be created in C++. However the vast majority is accessible, enabling you to easily and efficiently develop the functionality of your documents.
A good place to get started is the
luainvaders sample included with the library, which demonstrates many of the functionalities of the Lua plugin.