In the last post in the series on algorithmic feeds in Ktistec, I introduced the built-in criteria judge, which renders verdicts on posts based on the presence or absence of terms, like hashtags and keywords.

In this post, I introduce custom algorithmic feeds by showing you what a simple implementation of the criteria judge looks like if rewritten as a custom judge. Custom judges are nothing more than scripts, external to the server, that the server runs to judge candidate posts.

Here is the custom criteria judge:

#!/usr/bin/env python3
# ktistec: batch-size=250
# ktistec: param=any required
# ktistec: param=all
# ktistec: param=none
import html, json, re, sqlite3, sys
def terms(value):
    return [t.strip().lower() for t in (value or "").split(",") if t.strip()]
db = sqlite3.connect(sys.argv[1])
params = json.loads(db.execute("SELECT params FROM feed").fetchone()[0])
any_terms = terms(params.get("any"))
all_terms = terms(params.get("all"))
none_terms = terms(params.get("none"))
posts = db.execute("SELECT iri, name, summary, content, hashtags, mentions, labels FROM posts")
for iri, name, summary, content, hashtags, mentions, labels in posts.fetchall():
    text = " ".join(filter(None, (content, summary, name)))
    text = html.unescape(re.sub(r"<[^>]+>", " ", text)).lower()
    hashtags = {h["name"].lower() for h in json.loads(hashtags)}
    mentions = {m["name"].lower() for m in json.loads(mentions)}
    labels = {label.lower() for label in json.loads(labels)}
    def hit(term):
        if term.startswith("#"):
            return term[1:] in hashtags
        if term.startswith("@"):
            return term[1:] in mentions
        if term.startswith("§"):
            return term[1:] in labels
        return term in text
    included = not any(map(hit, none_terms)) and any(map(hit, any_terms)) and all(map(hit, all_terms))
    db.execute("INSERT INTO answers (iri, included) VALUES (?, ?)", (iri, int(included)))
db.commit()

It's simple, and it's not a perfect work-alike, but it shows the overall structure.

The Ktistec server runs judges, both built-in and custom, when there are candidate posts. Given a batch of posts, the judge makes exactly one decision: is each post included in the feed or excluded from the feed?

To do this, a judge is given the path to a temporary SQLite file (in sys.argv[1] in the example above) with information about the posts in the batch and their authors, as well as information about the feed itself.

Most of the time, posts arrive one at a time and the script runs on each post as it arrives. But the SQLite database can contain multiple posts—and it will when previewing or backfilling a feed—so the script must iterate over every post and judge each post independently.

In this example, the judge strips the post of markup and then looks for terms that match. It writes a verdict on the post into the answers table.

The front matter at the top of the file tells the server some things about the judge: the maximum batch size (the default is 1), and what params the judge expects the feed to provide. In this case, it expects three params, and the first is required. The params allow a user to create multiple, parameterized feeds from a single judge.

The params appear as inputs on the feed editor page.

The "Judge" dropdown at the top of the form only appears when you create a feed, and only when there are custom judges. You make a script a custom judge by placing the script in the <root>/etc/judges directory, making it executable, and waiting a few minutes until the server picks it up, or restarting the server. Here the custom criteria judge was saved as <root>/etc/judges/criteria-alike.

The example is in Python, but you can use the language of your choice.

In the next post, I'll go over what else in the SQLite database supplied to the judge is useful for judging.

#ktistec #AlgorithmicFeed #ActivityPub #fediverse