Release v2.4.6 of Ktistec is out. As mentioned in an earlier post, this release focuses on database performance improvements. This means caching the results of expensive queries (like counting all posts with a particular hashtag or mention). On my instance at least, pages like the notifications page are now snappier.

There are still slow queries (queries that take more than 50msec). Most of those are requests for pages of old posts where none of the necessary database pages are in the page cache. I have increased the page cache size, and that reduces the frequency, but I don't see an immediate fix.

Fixed

• Add missing database query logging.

Changed

• Improve query performance for hashtags and mentions.
• Make less costly updates to tag statistics.
• Improve anonymous session management.
• Cache the Nodeinfo count of local posts.

Removed

• Remove support for X-Auth-Token.

Other

• Add timeout values for POST socket operations.

I don't have an immediate plan for the next release. There have been a bunch of feature requests that I think have merit. I'll probably get started on some of those.

#ktistec #fediverse #activitypub #crystallang

You can specify the SQLite database and pass options (pragmas like cache_size, journal_mode, ...) on the command line when you start the Ktistec server. The following example sets the cache size to 20,000 pages (up from the default of 2,000 pages) which improves performance on larger instances.

KTISTEC_DB=~/ktistec.db\?cache_size=-20000 ./server

You can also enable the write-ahead log (but make sure you know what that means).

KTISTEC_DB=~/ktistec.db\?journal_mode=wal\&synchronous=normal ./server

Pragmas supported are limited to those listed here.

#ktistec #fediverse #activitypub #crystallang

A new release of ktistec that improves database performance is imminent. In the past, database optimization usually meant "fixing a bunch of poorly constructed queries", and I'm sure there's more of that to do—I'm not an expert. But this time, I found most of the queries were as good as they were going to get on my watch (I'm not an expert). If you have a million records and you need to filter and count them, that's just going to take some time...

So this time, I focused on caching the results of queries like that (which really means I focused on cache invalidation, right). A case in point is commit d544b1af. Previously, the nodeinfo endpoint filtered and counted posts on every request, and it took +80msec to do that. Worse, the filtering pushed everything else out of the sqlite page cache, which made the next, unrelated database query slow!

Caching this value, and only recounting when I post something, not only dropped the service time for the request to ~1msec but actually improved database performance, generally!

More to come...

#ktistec #fediverse #activitypub #crystallang

If you're running an instance of Ktistec and want to see what other ActivityPub instances are sending you, turn on JSON-LD processing debug logging.

1. Go the the /system URL.
2. Find the ktistec.json_ld setting.
3. Select "Debug" and save.

Ktistec will dump received activities to the log, after the activity has been parsed into JSON but before JSON-LD expansion.

2025-01-22 14:53:17 UTC 409 POST /actors/toddsundsted/inbox 4.29ms
2025-01-22T14:53:17.597172Z  DEBUG - ktistec.json_ld: {"@context" => ["https://www.w3.org/ns/activitystreams", "https://w3id.org/security/v1"],
"id" => "https://random.site/users/FooBar#delete", "type" => "Delete", "actor" => "https://random.site/users/FooBar", "object" => "https://random.site/users/FooBar", "to" => ["https://www.w3.org/ns/activitystreams#Public"], 
"signature" => {"type" => "RsaSignature2017", "creator" => "https://random.site/users/FooBar#main-key", "created" => "2025-01-22T14:52:40Z", "signatureValue" => "01234567890abcdefghijklmnopqrstuvwxyz=="}}

Answer to a FAQ:
The server returns HTTP status code 409 ("Conflict") if it has already received an activity.

#ktistec #fediverse #activitypub

Ktistec release v2.4.5 rolls out the build time and executable size optimizations I've been blogging about here. It also fixes a few small bugs.

Fixed

• Handle @-mentions with hosts in new posts.
• Handle HEAD requests for pages with pretty URLs.
• Destroy session after running scripts.

Changed

• Delete old authenticated sessions.

I've started a branch full of query optimizations. My general rule—as highlighted in the server logs—is if a query takes longer than 50msec, it takes too long. It's time to address some problems...

#ktistec #fediverse #activitypub #crystallang

Ktistec release v2.4.4 fixes a few things in the prior release and introduces at least one killer feature!

Fixed

• Always get the attachments. (fixes #119)
• Don't run scripts until the server has been configured.

Changed

• ⭐️ Make the editor toolbar sticky.
• Clear the cached translator when the settings change.

I'm spending some cycles looking at the size of the server executable. You can read about my approach to reducing Crystal Language executable size and build time here.

#ktistec #fediverse #activitypub #crystallang

Ktistec release v2.4.3 supports language translation.

Inspiration for this feature comes from Mastodon.

In order to enable translation, you need an API key for either DeepL or LibreTranslate. These are the only services Ktistec supports at this time.

Posts from properly configured accounts on supported servers, like Mastodon, include the content language. On posts like these, Ktistec will display a button to translate the content if the language differs from your language.

Unfortunately, not all Fediverse/ActivityPub servers explicitly support language (I mean, Ktistec didn't until just now). And not all users correctly set their posts' language, so ymmv... but it has been hugely useful for me.

I'm going to focus on site customization next (colors, etc.).

#ktistec #fediverse #activitypub #crystallang

Release v2.4.2 fixes a few more bugs. Only one is a regression—I found the others while testing. In this release:

Fixed

• Fix metrics chart line labels.
• Permit del, ins, and s elements in sanitized HTML.
• Only store the redirect_after_auth_path on browser navigation.
• Add "content" property to editor JSON error response.
• Use FileUtils.mv to move uploaded files. (fixes #117)

Thanks to @jayvii for help with troubleshooting the last one!

#ktistec #fediverse #activitypub #crystallang

Eh, I didn't test enough and released v2.4.0 of Ktistec with a few annoying regressions. Release v2.4.1 fixes them! ☹️ 😠 😡 🤬

Fixed

• Handle edge cases and race conditions in script runner.
• Reconnect editor autocomplete support.
• Fix permissions on uploaded files.

#ktistec #fediverse #activitypub #crystallang

I just released v2.4.0 of Ktistec. This release encompasses a few things that I've been working on for a while: improved support for operating without JavaScript available/enabled and support for running scripted automations.

Except for a few items, Ktistec now works without JavaScript. Obviously, things like WYSIWYG editing of HTML don't work—I plan to add support for Markdown to compensate. Running in Lynx is a stretch, but...

Since the early days, most controller actions supported both text/html and application/json.  I cleaned up support for the latter and have officially documented the Ktistec API in the README.

In addition, I've added support for running bots/automations (prior announcement). The Ktistec server will periodically run any executable script in the etc/scripts directory. These scripts have access to the Ktistec API and can post, follow, share, like, etc. This is experimental and obviously introduces an attack surface, though that shouldn't be a problem on correctly configured hosts.

Here's the full changelog:

Added

• Support running scripted automations.
• "Fetch Once" button on hashtag and thread pages. (fixes #108)
• Support navigation to a post's threaded view. (fixes #108)
• Add support for post name and summary.

Fixed

• Improve support for operating without JavaScript available/enabled.
• Only enforce CSRF protection on "simple" requests.

Changed

• Replace use of multi-action controller with formaction. (fixes #101)

Other

• API usability improvements.

#ktistec #fediverse #activitypub #crystallang