Building OpenPanel Modules

From Openpanel Documentation Wiki
Jump to: navigation, search
Important note: Some information on this page may be outdated. This note will be removed if the content has been verified to apply to the 1.0 release.

Most of the documentation on this site (and thus, this page) applies to 1.0. By far the most important change is that opencli is now called openpanel-cli.

OpenPanel modules are extensions that allow you to control provisioning and parameters for any extra services you want to add. In fact, all the services OpenPanel already controls are implemented through the same module interface. As a result, you can create modules that replace existing built-in functionality as easily as you can create pure extensions.

Modules can be written in any language, with calling conventions using either two-way XML packets or parameter passing through crafted environment variables. Supplementary API libraries have been released for Python, C++ and shell scripts. This document will shed some light on the inner workings of the OpenPanel system and the decisions you need to make when you’re considering creating your own module.

Contents

The Object Model

Through the opencore service, OpenPanel exposes an architectural view of objects containing other objects and data elements. Individual modules take care of implementing one or more classes within this model. The User class is sort of special, in so far as that it is a required class that is used for tracking the ownership of individual objects.

Bom diagram1.png
In the illustration to the right we show the layout of these two classes as implemented by User.module and the SSH:Shell class as defined by SSH.module. Note that the class for SSH:Shell is defined as a singleton. Only one instance may exist within context and it will always get the same id, in this case ‘shell’.

The User class can also own other User objects, which makes it possible to deal with the concept of resellers and sub-admins. All normal objects have a hidden owner field that links to a User. Permission to view and edit objects always depends on the actual heritage of the User object that is associated with the currently logged in OpenPanel user.

If you are creating a module for a completely uncovered service, chances are your module’s objects will live either under a User or a Domain object. An exception is formed by modules that manipulate system-wide settings; they may export classes at the root level. Also, if you’re dealing with services that are require an existing service like a website or a mail domain, your class may actually live underneath a class like Mail.

The easiest interaction model for your module is if you can consider each object within its own context and, for example, create a distinct file for each one of them. This is usually possible when dealing with subsystems like Apache that have provisions for configuration directories that can be included in the config ad verbatim. For subsystems with a monolithic configuration, there are ways to pass more complete data to the module, as we will illustrate later in this document.

Modules in Context

Here is a view of a couple of more classes in context. Classes with the same color fall within the same module. when you want to come up with a good strategy for placing your new classes, it is wise to study the classes and relationships as they exist within OpenPanel

Bom moreclasses.png

Also, you'll need to decide on the indexing properties of your class. Most of the times it is convenient to define a key space for object-id's, either one that is global (for instance, domain names), context-local or irrelevant (like in the case with DNSDomain:Record objects under the DNSDomain module).

Module Components

An OpenPanel module is basically a directory with a '.module' suffix inside the /var/openpanel/modules directory. It contains, at a minimum, the following items:

Bom modlayout.png
action 
Implementation part of the module, handles object mutations and other behavior.
verify 
Determines whether the environment is sane for running the module (for instance, if the module controls postfix it should make sure postfix is configured correctly for understanding the extra configuration files the module will install).
module.xml 
An xml headerfile that defines the classes and options associated with the module.
html files 
Object classes that need a more thorough definition in the GUI can define a html file to be included in module.xml.
png files 
Objects that appear within their own category in the CrowBar will define an icon file.

These are the only files openpanel cares about, the rest is completely up to your module’s design.

A Simple Module Implementation: the 'Storpel' service

For a quick feel of things, we will take a look at a simple module design for the imaginary Storpel service.

We will be using a somewhat naive security model: All configuration files for our service will be kept in a directory that is directly writable by the opencore group. This way we can skip any privileged operations and keep a relatively simple design. The Storpel service loads per-user files out of the /etc/storpels directory. We will create this directory on the system and give it the proper ownership flags:

Bom term mkdir storpels 1.png

The Scripts

First let's create the action script using the bash API. This script implements the create, update and delete methods for our class.

#!/bin/bash
. /var/openpanel/api/sh/module.sh

Storpel.create() {
  model=$(coreval Storpel model)
  id=$(coreval OpenCORE:Session objectid)
  echo "model=${model}" > "/etc/storpels/${id}.storpel"
}

Storpel.update() {
  Storpel.create
}

Storpel.delete() {
  id=$(coreval OpenCORE:Session objectid)
  rm -f "/etc/storpels/${id}.storpel"
}

implement Storpel.module

As you can see we’re lazy with updates and just rewrite the file like we do for a new file. The verify script is even simpler:

#!/bin/sh
if [ ! -d /etc/storpels ]; then
  echo "Missing /etc/storpels"
  echo "quit" >&3
  exit 1
fi
echo "quit" >&3

We just check for the /etc/storpels directory, if it doesn’t exist we send a complaint. That will prevent the module from being activated in opencore.

Generating module.xml

There are two ways to create your module.xml file. You can either create the XML directly, or you can create a module.def file and use the mkmodulexml tool to convert this into an xml file during your build process. Let's take a look at both. This is the module.def for Storpel.module:

module Storpel                < uuid b7b26fd8-6996-4c1b-877f-a0fafd397a58
                              < version 1.0
                              < languages en_EN
                              < apitype commandline
                              < getconfig false
                              
class Storpel                 < uuid 5437797e-7a3e-46d8-8bb1-d43ec457e18a
                              < version 1
                              < indexing manual
                              < requires User
                              < shortname storpel
                              < description Storpel Installation
                              < uniquein class
                              < capabilities create delete update
                              
    string model              : Model
                              < default basic

When running this through mkmodulexml, this will be the output:

<?xml version="1.0" encoding="UTF-8"?>
<com.openpanel.opencore.module>
    <name>Storpel.module</name>
    <uuid>b7b26fd8-6996-4c1b-877f-a0fafd397a58</uuid>
    <version>1.0</version>
    <languages en_EN=""/>
    <implementation>
        <apitype>commandline</apitype>
        <getconfig>false</getconfig>
    </implementation>
    <classes>
        <class id="Storpel">
            <uuid>5437797e-7a3e-46d8-8bb1-d43ec457e18a</uuid>
            <version>1</version>
            <indexing>manual</indexing>
            <requires>User</requires>
            <shortname>storpel</shortname>
            <description>Storpel Installation</description>
            <title>Storpel</title>
            <uniquein>class</uniquein>
            <capabilities create="true" delete="true" update="true"/>
            <parameters>
                <p id="model" enabled="true" visible="true" required="true" type="string" default="Basic">Model</p>
            </parameters>
        </class>
    </classes>
</com.openpanel.opencore.module>

With this installed in /var/openpanel, we should have a perfectly working module. This is what it will look like in the GUI:

Bom storpel1 in gui.png


and through openpanel-cli you will see something like this:

Bom storpel1 terminal.png

The data that was sent to the module for creating the object would look like this in XML:

<?xml version="1.0" encoding="UTF-8"?>
<dict>
	<string id="OpenCORE:Context">Storpel</string>
	<dict id="OpenCORE:Session">
		<string id="sessionid">2c6e2b80-ebb3-4a44-925a-458b7e13f4a4</string>
		<string id="classid">Storpel</string>
		<string id="objectid">mainstorpel</string>
	</dict>
	<dict id="Storpel" type="object" owner="">
		<string id="model">Express</string>
		<string id="uuid">74021559-5823-4a5e-15b6-c3574d1c9946</string>
		<string id="version">1</string>
		<string id="parentid">25ec7e4a-59e3-43a6-3b30-d9b41d258a0f</string>
		<string id="id">mainstorpel</string>
		<string id="metaid">mainstorpel</string>
	</dict>
	<dict id="User" type="object" owner="">
		<string id="password">$1$C4jlyBnx$EkPEfJkEKtETp0UcJcSPu0</string>
		<string id="name_customer">Administrator</string>
		<string id="language">en_EN</string>
		<string id="emailaddress">openadmin@example.net</string>
		<string id="uuid">25ec7e4a-59e3-43a6-3b30-d9b41d258a0f</string>
		<string id="version">2</string>
		<string id="id">openadmin</string>
		<string id="metaid">openadmin</string>
	</dict>
	<string id="OpenCORE:Command">create</string>
</dict>

Performing Privileged Commands

The naive approach of giving the opencore user or group access to your system configuration files may get you started easily, but will leave you vulnerable to programming errors in any loaded module.
Bom authd.png

The authd service can help your module with performing more privileged operations (as the root user, or with a more specific target uid/gid).

Authd is the only process that runs with root privileges. It keeps a unix domain socket open that can be reached by processes that carry the authd group, in this case opencore. Before spawning and executing a module’s action or verify scripts, opencore will open a connection to this authd socket, pre authenticate it against the module that will be run and drop the authd group before traversing into the module domain. The authd service waits for commands from the module, validating them against enumerated privileges in the module’s module.xml file. If the requested operation falls within these defined boundaries, the privileged operation will be executed.

Most authd operations are related to fishing files out of the module-writable staging directory and getting them installed in privileged system directories, but it can also be made to run specific scripts out of the tools directory in /var/openpanel or to manipulate the unix user database. By carefully enumerating the desired privileged operations, you protect your module from being subverted into messing with files and directories it should not be touching.

The Updated Module

The bash API implements the authd command to communicate with authd. We'll use this to install our .storpel files as the root user.

#!/bin/bash
. /var/openpanel/api/sh/module.sh

Storpel.create() {
  model=$(coreval Storpel model)
  id=$(coreval OpenCORE:Session objectid)
  echo "model=${model}" > "${id}.storpel"
  authd installfile "${id}.storpel" /etc/storpels
}

Storpel.update() {
  Storpel.create
}

Storpel.delete() {
  id=$(coreval OpenCORE:Session objectid)
  try_authd deletefile "/etc/storpels/${id}.storpel"
  rm -f "${id}.storpel"
}

implement Storpel.module

Instead of writing directly to /etc/storpels, we now store the files inside Storpel's staging directory. The use of try_authd inside Storpel.delete turns an error return from authd into a soft failure, it's a good habit not to fail unnecessarily when something goes wrong in the delete stage.

The New module.def

The module.def stays mostly the same, but we'll need to add a section to the bottom that will tell authd which operations are allowed:

authd fileops
    match *.storpel           : /etc/storpels
                              < user root
                              < group root
                              < perms 0644

The fileops section defines access to /etc/storpels for files in the module's staging directory matching the provided pattern. With this new version installed, we should be able to keep a root-only /etc/storpels directory:

Bom storpel2 terminal.png

Enumerated Types

One thing we've not been handling too elegantly so far either is the completely free-form 'model' field we defined for our Storpel object. In reality, there's a limited set of choices. You can define an enum section inside module.def to create such a list and linking it to the field:

class Storpel                 < uuid 5437797e-7a3e-46d8-8bb1-d43ec457e18a
                              < version 2
                              < indexing manual
                              < requires User
                              < shortname storpel
                              < description Storpel Installation
                              < title Storpel
                              < uniquein class
                              < capabilities create delete update
                              
    enum model                : Model
                              < default basic

enum model
    value basic               : Basic model
    value expert              : Expert model
    value classic             : Classic model

Multiple classes in your module.def can refer to the same enum, provided they use the same field key for this enum parameter. Changes to the enum-configuration only have an effect on acceptable values from the user interface; the database itself does not enforce these values and will keep on listing values that may, in a new version of your enum, no longer be valid. The module is expected to deal with this gracefully.

Adding a System Configuration Class

As a next logical step for our module we may want to allow some of the global settings of the Storpel service to be changed through OpenPanel:

Bom storpeld conf.png

For this we will introduce a new class StorpelService which will live at the root level of the object hierarchy as a singleton object. Let's look at the complete module.def with this class added:

# ============================================================================
# Storpel.module (c) 2008 Acme Inc.
# ============================================================================
module Storpel                < uuid b7b26fd8-6996-4c1b-877f-a0fafd397a58
                              < version 1.0
                              < languages en_EN
                              < apitype commandline
                              < getconfig false
                              
# ============================================================================
# CLASSES
# ============================================================================
class Storpel                 < uuid 5437797e-7a3e-46d8-8bb1-d43ec457e18a
                              < version 2
                              < indexing manual
                              < requires User
                              < shortname storpel
                              < title Storpel
                              < description Storpel Installation
                              < uniquein class
                              < capabilities create delete update
                              
    enum model                : Model
                              < default basic

# ----------------------------------------------------------------------------
class StorpelService          < uuid 9e1c09e6-281e-403f-ae8e-e8d700b892e0
                              < version 1
                              < indexing manual
                              < shortname storpeld
                              < title Storpel Service
                              < description Global configuration for Storpels.
                              < uniquein class
                              < singleton storpeld

    integer listenport        : TCP Listening Port
    string servername         : Advertised server name
                              
# ============================================================================
# ENUMS
# ============================================================================
enum model
    value basic               : Basic model
    value expert              : Expert model
    value classic             : Classic model

# ============================================================================
# AUTHD
# ============================================================================
authd fileops
    match *.storpel           : /etc/storpels
                              < user root
                              < group root
                              < perms 0644
                              
    match storpeld.conf       : /etc
                              < user root
                              < group root
                              < perms 0644

authd services
    service storpeld

The most notable part of the StorpelService class definition is the singleton value. It declares that there should only be one instance of this class active in context and it should get the automatic id of 'storpeld'. There's also a bit more action with regards to authd: We grant our module access to the storpeld service, which means we will be able to ask authd to send a reload to its init-script.

Changes to the Script

Now let's look at the code for handling the StorpelService class. We'll insert these before the "implement Storpel.module" statement:

StorpelService.create() {
  listenport=$(coreval StorpelService listenport)
  servername=$(coreval StorpelService servername)
  cat > storpeld.conf << _EOF_
[system]
ListenPort = $listenport
ServerName = $servername
_EOF_
  authd installfile storpeld.conf /etc
  authd reloadservice storpeld
}

StorpelService.update() {
  StorpelService.create
}

StorpelService.delete() {
  exiterror "Delete not implemented"
}

We use the same simple approach towards updates and we'll skip on implementing delete, which makes little sense in this context anyway. If we try it out like this, we'll encounter a little oddity though: The configuration, as far as OpenPanel is concerned, does not exist until we create it for the first time:

Bom storpel3 terminal.png

After the initial create, the module works as intended: It can only be updated, not deleted - and no other objects can be created. We'll look at polishing the create problem next.

Importing Pre-existing Configuration

Generally, your module is supposed to work on a service that already has a working configuration at the time your module is installed. To get install-time configuration into the opencore database we need to get opencore to use the getconfig call into the module. Whenever opencore runs into a module for the very first time, it will use this call to load the pre-existing configuration into the openpanel database. To activate this behavior, change the boolean getconfig value in module.def from false to true.

On the script side we need a little bit more work — opencore expects XML-encoded data back for this call containing a list of all pre-existing objects it should import. We'll use some old-fashioned ugly grep-and-sed work to get the info out of the storpeld.conf configuration file.

Module.getconfig() {
  listenport=$(cat /etc/storpeld.conf | grep ListenPort | sed -e "s/.* = //")
  servername=$(cat /etc/storpeld.conf | grep ServerName | sed -e "s/.* = //")
  cat << _EOF_
  <dict id="StorpelService" type="class">
    <dict id="storpeld">
      <integer id="listenport">$listenport</integer>
      <string id="servername">$servername</string>
    </dict>
  </dict>
_EOF_
 exitquiet
}

Now the storpeld configuration block will be available to OpenPanel users immediately after the module is installed.

Cleaning Module Data

If your module is under active development, there may be times that you want to update the field layout of your classes and test the module mostly afresh. The best way to go about this is to use the coreunreg tool:

Bom coreunreg.png

This tool will remove references to your module and its classes from opencore's internal databases. On renewed start-up, opencore will treat the module as completely new (including running getconfig).

Using the Domain and Domain:Alias classes

Let's make things more interesting and define Storpels to be bound to a domainname, much like a virtual host object or mail domain. For many services this approach tends to make more sense. The new situation will look something like this:

Bom storpel aliaslayout.png

The major new issue to take into account here is the situation with Domain:Alias objects. The Domain class gives this class special status: Requests for other classes that live under the Domain class will also contain the list of associated Domain:Alias objects, put inside the already normally provided data about the Domain object that is the actionable object's parent.

Let's assume that the storpel service also understands the concept of defining aliases and that a .storpel file in /etc/storpels will look something like this:

Bom storpel aliasconfig.png

Apart from the enumeration of aliasdomains alongside the hostname, we're also using a more strict schema for the object-ids now, we want them restricted to being the same as (or a subdomain of) the parent Domain.

The New module.def

The only thing that changes here is the class definition for Storpel:

class Storpel                 < uuid 5437797e-7a3e-46d8-8bb1-d43ec457e18a
                              < version 2
                              < indexing manual
                              < requires Domain
                              < shortname storpel
                              < description Storpel Installation
                              < title Storpel
                              < uniquein class
                              < parentrealm domainsuffix
                              < capabilities create delete update

We changed the parentage by setting requires to 'Domain' and added a new tag parentrealm to enforce the new restrictions on a Storpel's id.

The New Script

For the script, we only need to alter the Storpel.create function:

Storpel.create() {
  model=$(coreval Storpel model)
  id=$(coreval OpenCORE:Session objectid)
  echo "Model $model" > "${id}.storpel"
  echo "Hostname $id" >> "${id}.storpel"
  listaliases | while read a; do
    echo "Alias $a" >> "${id}.storpel"
  done
  authd installfile "${id}.storpel" /etc/storpels
}

The major change here is that we use the listaliases function to get a list of mutated alias-ids, depending on the existing Domain:Alias objects. This is just a convenience function in the bash API, if you're interacting with opencore through XML with another language, it may be useful to understand what it does, so here it is:

listaliases() {
  mainDomain=$(coreval Domain id)
  targetDomain="$OBJECTID"
  coreval --loop Domain Domain:Alias | while read aliasDomain; do
    echo "$targetDomain" | sed -e "s/${mainDomain}/${aliasDomain}/"
  done
}

The end-result is that we get a neat list of alias hostnames alongside our primary hostname:

Bom storpelwithaliases.png

In XML the request looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<dict>
	<string id="OpenCORE:Context">Storpel</string>
	<dict id="OpenCORE:Session">
		<string id="sessionid">445e23b4-4290-48df-5fab-67584fb62825</string>
		<string id="classid">Storpel</string>
		<string id="objectid">my.magicstorpel.net</string>
	</dict>
	<dict id="Storpel" type="object" owner="openadmin">
		<string id="model">basic</string>
		<string id="uuid">66fa83ab-4c95-4de7-7433-44f5200e7512</string>
		<string id="version">1</string>
		<string id="parentid">2428058b-3f27-4fdd-7f13-9fe92a35f750</string>
		<string id="ownerid">25ec7e4a-59e3-43a6-3b30-d9b41d258a0f</string>
		<string id="id">my.magicstorpel.net</string>
		<string id="metaid">my.magicstorpel.net</string>
	</dict>
	<dict id="Domain" type="object" owner="openadmin">
		<string id="uuid">2428058b-3f27-4fdd-7f13-9fe92a35f750</string>
		<string id="version">1</string>
		<string id="ownerid">25ec7e4a-59e3-43a6-3b30-d9b41d258a0f</string>
		<string id="id">magicstorpel.net</string>
		<string id="metaid">magicstorpel.net</string>
		<dict id="Domain:Alias" type="class">
			<dict id="magicstorpel.com" type="object">
				<string id="class">Domain:Alias</string>
				<string id="uuid">72786d0f-6f33-4f48-3ea2-8f3d14b05f57</string>
				<string id="parentid">2428058b-3f27-4fdd-7f13-9fe92a35f750</string>
				<string id="ownerid">25ec7e4a-59e3-43a6-3b30-d9b41d258a0f</string>
				<string id="id">magicstorpel.com</string>
				<string id="metaid">magicstorpel.com</string>
			</dict>
			<dict id="magicstorpel.org" type="object">
				<string id="class">Domain:Alias</string>
				<string id="uuid">389df3fc-9421-4697-722b-1c1b660198e1</string>
				<string id="parentid">2428058b-3f27-4fdd-7f13-9fe92a35f750</string>
				<string id="ownerid">25ec7e4a-59e3-43a6-3b30-d9b41d258a0f</string>
				<string id="id">magicstorpel.org</string>
				<string id="metaid">magicstorpel.org</string>
			</dict>
		</dict>
	</dict>
	<string id="OpenCORE:Command">create</string>
</dict>

A side-effect of this approach is that your module will get an update trigger whenever an alias is added or removed from the domain.

Monolithic Configuration Files

In situations where you have an object of class A that can contain a variable number of related objects of class B, you may be restricted by the configuration format in such a way that you need to know about all the B objects whenever you're writing to this configuration. Let's say a Storpel instance can have a list of authenticated users like this:

Bom storpelwithusers.png

First let's define the new StorpelUser class in module.def:

class StorpelUser             < uuid 27c993d6-3475-4ff4-a2fc-45d15cd4b28a
                              < version 1
                              < indexing manual
                              < requires Storpel
                              < title Storpel User
                              < shortname user
                              < description A user that can log into the Storpel
                              < uniquein parent
                              < capabilities create delete update

                              
    string password           : Password

Also, we'll need to flag the opencore that information about the Storpel class should always be accompanied by its child objects:

class Storpel                 < uuid 5437797e-7a3e-46d8-8bb1-d43ec457e18a
                              < version 3
                              < indexing manual
                              < requires Domain
                              < shortname storpel
                              < description Storpel Installation
                              < title Storpel
                              < uniquein class
                              < parentrealm domainsuffix
                              < allchildren true
                              < capabilities create delete update
                              
    enum model                : Model
                              < default basic

A side-effect of setting allchildren to true is that there really isn't much of a distinction between actions on Storpel and those on StorpelUser, when it comes to the data sent to the module. This makes our handler functions for StorpelUser pretty straightforward:

StorpelUser.create() {
  Storpel.update
}

StorpelUser.update() {
  Storpel.update
}

StorpelUser.delete() {
  Storpel.update
}

The harder part is rewriting the handler for Storpel to take the enclosed StorpelUser records into account:

Storpel.create() {
  model=$(coreval Storpel model)
  id=$(coreval Storpel id)
  echo "Model $model" > "${id}.storpel"
  echo "Hostname $id" >>"${id}.storpel"
  listaliases | while read a; do
    echo "Alias $a" >> "${id}.storpel"
  done
  coreval --loop Storpel StorpelUser | while read userid; do
    password=$(coreval Storpel StorpelUser "$userid" password)
    echo "User ${userid}:${password}" >> "${id}.storpel"
  done
  authd installfile "${id}.storpel" /etc/storpels
}

Like in the earlier illustration of listaliases, we use the coreval tool to query for all StorpelUser objects inside the provided Storpel object.

Using Crypted Passwords

Our current implementation of plaintext password is, of course, an abomination. Luckily, opencore can create salted MD5 or DES hashes for you, if you ask nicely:

class StorpelUser             < uuid 27c993d6-3475-4ff4-a2fc-45d15cd4b28a
                              < version 1
                              < indexing manual
                              < requires Storpel
                              < title Storpel User
                              < shortname user
                              < description A user that can log into the Storpel
                              < uniquein parent
                              < capabilities create delete update

                              
    password password         : Password
                              < crypt md5

by changing the field type from string to password and adding a crypt attribute, we tell opencore to pre-crypt the data for the module.

Personal tools
Namespaces
Variants
Actions
Documentation
Tools
Toolbox