tPSGI

tPSGI

tpsgi ... [starman args]

A PSGI server built to migrate old mod_perl/apache stacks onto psgi, and that powers tCMS.

Will execute any executable file (chmod +x) in www/ as CGI, and otherwise host any static file therein.

The idea here is as a bridge between old and new which allows you to modernize at your own pace, gradually turning CGIs into routes.

The other idea is that we don't want to add any extra abstraction if we don't have to, while gently guiding into better practices. Shops used to having full control over the HTTP response should continue to do so, rather than being locked into one framework or another. This allows them to continue doing whatever funky thing they have been doing, like COMET, et cetera. The best way to do that is to expect raw PSGI to be output.

Configuration

Obeys ~/.tpsgi.ini -- this is what will control all the custom aspects mentioned below:

verbose     : Whether to print all log messages or not
custom_log  : Custom location for default (rotated) log
routers     : Router module(s), separate with comma.  Relative to TPSGI install dir.
indices     : Dirindex files.  In addition to index.html, index.htm, index.cgi
loggers     : Logger module(s), separate with comma
auth        : Authentication module
domain      : Domain name of the application, used by things like service files
basedir     : Directory you want to chdir into on startup. Relative to TPSGI install dir.  Default '.'
user        : Who to run as
http_user   : Who's proxying this application (used for statics)
autoreload  : Whether or not to automatically reload the application when library files change. Default 0.

You can run bin/tpsgi-config to get any config value needed by scripts.

We wrap starman with a custom script, bin/tarbaby that implements Plack::Loader::Reload far less dangerously. It only reloads the code relevant to tPSGI and your application itself. All the CPAN deps should be updated out-of-band and the service manually restarted, as this is far more fraught.

Custom routing

All passed routing modules will be required, and combined to form a routing table so you have an analogue to rewrite rules.

Routing Modules look like so:

our @routes = (
    '/foo/(\w+)/save' => {
        method    => 'GET',
        callbacks => { # Run these subroutines to execute the appropriate handler given the content-type.  Expects normal PSGI output.
            'text/html'        => \&render,
            'application/json' => \&dump,
        },
        noindex   => 0, # Explicitly exclude this in robots.txt
        nomap     => 0, # Don't put this in the sitemap
        acls      => ['admin'], # Route requires both authentication by auth handler and the auth'd user to have the provided ACLs.
        data      => { baz => 'throb' }, # Arbitrary data to inject into the GET/POST data hashref
        captures  => ['name','parameter'], # Name parameters of the capture groups in the route key.  Suppose we want /foo/don/keys here.
        static    => 1, # Whether we can save a static render of this
        invalidates => ['/foo/(\w+)'], # Static renders which are invalidated by this route returning a successful response code.
    },
    ...
);

Note that this is an array, as we want this to preserve order. Fat commas are used to hint that this is in fact a tuple.

Router callbacks are passed the TPSGI object and the raw query, which means you can do:

sub route {
    my ($self, $query) = @_;
    $self->INFO("whatever");
    $self->ERROR("eeee");
    $self->DEBUG("blah");

    # Either from the GET or POST data, POST is preferred when both are present.
    my $param = $query->{param};

    my %headers = ...;

    return [200, \%headers, ["Hello world"]]
}

Watching files for changes

tPSGI already watches your library directories with inotify so that it can reload itself when your code changes. Your application can put arbitrary files and directories onto that same watchlist, along with a callback to run when they change. The usual reason to want this is throwing away a static render because the thing it was generated from changed underneath it.

Routing modules declare these the same way they declare routes:

our %watches = (
    # A bare coderef is the common case: run this when the path changes.
    'data/posts.json' => sub {
        my ($tpsgi, $change) = @_;
        $tpsgi->invalidate_renders('html');
    },

    # Or spell it out if you want to be picky about which events you care about.
    'www/assets' => {
        callback => \&asset_changed,
        events   => [qw{CREATE MOVE}],
    },
);

...or register them at runtime with $tpsgi->add_watch($path, $callback, %options), and drop them again with $tpsgi->remove_watch($path).

Callbacks are passed the TPSGI object and a hashref describing what happened:

sub asset_changed {
    my ($tpsgi, $change) = @_;

    $change->{path};    # full path of the thing which changed
    $change->{watched}; # the path the watch was registered against
    $change->{events};  # arrayref of event names, e.g. ['MODIFY']

    $tpsgi->invalidate_render($change->{path}, 'html');
}

A few things worth knowing:

• Watches are shared by every worker, so whichever worker notices a change first consumes it for all of them. Declare your watches in your routing module (which every worker loads) rather than from inside a single route, and make the callbacks safe to run in any worker.
• Nothing sits in a select loop over the inotify descriptor, so changes are noticed when the next request comes in. Prompt, but not instant.
• Watching the containing directory is more robust than watching a single file. Watches on files which get replaced wholesale (write a tempfile, rename over the target) are re-established afterwards, but a file which is deleted and not recreated takes its watch with it.

Custom Authentication handling

The (optional) Authentication handler can be passed, and look like so:

package Auth::Handler;

# Is user who they say they are
sub authenticate {
    my (%auth_payload) = @_;
    ...
    return ($session_cookie);
}

# is the user authorized to do $thing
sub authorized {
    my ($user, $priv) = @_;
    ...
    return 1 || 0;
}

# Available ACLs
sub acls {
    ...
    return %acls;
}

# Dump acls for user
sub acls_for_user {
    my $user = shift;
    ...
    return %acls;
}

# What is the user for this session token
sub user_for_session {
    my $session = shift;
    ...
    return $user;
}

In the event that auth or route handlers are found, these events will be noted in the startup log.

Logging

By default we have two log handlers...which you can augment with your own, based on Log::Dispatch.

The first default is to print ERROR and worse to the STDOUT of the PSGI server. The other is to emit INFO or better to logs/tpsgi.log

Pass your own dispatch subclasses and watch it go whir.

Service configuration

Proper operation of this as a production service requires some things:

1. You have a user which has this reposity as their home directory.
2. You run the shell script, tpsgi.sh in service/ to run this as a service.
3. You set the username and the domain name of the service in .tpsgi.ini
4. You set your PATH appropriately to pick up on things like custom perls in a .bashrc
5. You set the group of your reverse proxy application in .tpsgi.ini (the http_user variable)

run service/tpsgi.sh as root and you should be set supposing the above is true.

In general the operation is like so:

1. nginx (or whatever) looks for a sock file owned $USER:$WEB_GROUP, and reverse proxies to this
2. tpsgi is actively listening on this sock file.

Generally you'll just run bin/build_service and then systemctl start $domain. It's configured to start as root, then drop privs thanks to Net::Server's capabilities inherited in starman.

Secrets the application needs

An application often needs a key or a password that has no business living in its own checkout, where it would go into every backup alongside whatever it protects. The unit imports those from systemd's credential store and service/tpsgi.sh hands them on.

Put one there, sealed to this machine so that the file is useless anywhere else:

openssl rand -base64 32 \
  | sudo systemd-creds encrypt --name=tcms-vault - /etc/credstore.encrypted/tcms-vault

...or unencrypted, if the machine has no TPM, in which case it is a root-owned file rather than a sealed one:

openssl rand -base64 32 | sudo tee /etc/credstore/tcms-vault > /dev/null
sudo chmod 600 /etc/credstore/tcms-vault

systemd decrypts it into a ramdisk at /run/credentials/$service that only this service can read. Nothing is written down in the unit, and nothing breaks if it is not there: ImportCredential= is quiet about a credential that does not exist, so an installation with no secrets to hand over starts normally.

The workers are chrooted into the application directory and that ramdisk is outside it, so service/tpsgi.sh reads the credential before the chroot happens and exports it -- tcms-vault becomes TCMS_VAULT_KEY. An application taking one of these is expected to read it out of its environment as it starts and delete it there and then, so that nothing it forks afterwards inherits it.