on a separate note, why am i discovering all of the things that don’t work with #ktistec while i am away!?! fetching just stopped for a few hashtags I follow…!

I just released v2.2.0 of Ktistec. A lot of code was written but the main user-visible change is streaming timeline updates to the web client using server-sent events.  The back-end pub/sub framework will also make it easier to decouple some components down the road.

I want a visual indicator that new posts are in my timeline. I appreciate that not everyone feels the same way—if there's interest I may make it switchable.

And now, off to #worldcon!

#ktistec #fediverse #crystallang

I just released v2.0.0 of ktistec. It's just bug fixes on top of the last prerelease, but it's much improved over v1.0.0. The README and CHANGELOG have more detail but features and improvements include:

• Content filtering
• Support for more ActivityPub object types
• Support for following threads, hashtags and mentions
• Collection and presentation of server performance metrics
• Setting the application name in the webmanifest (thanks JayVii)
• Mastodon-compatible user profile metadata (thanks relistan)
• Links to the pages of remote actors (thanks vrthra)
• Support for handling 303s in searches (thanks winks)
• Automatic database backups

I'm working on user experience enhancements for the next release: I've just finished streaming updates to the front-end—no more refreshing the page to see if you have new posts in your timeline (and new notifications).

#ktistec

the last few days are more in line with what i expect. there is a (temporary) bump in memory, but that corresponds to a large increase in traffic.

#ktistec #crystallang

for reasons i don't understand, some restarts of epiktistes result in huge memory usage spikes—reported both by the garbage collector and the operating system. what's interesting is heap size and free memory move roughly in tandem (though the difference isn't constant), which implies that there is free memory, but perhaps it's too fragmented to be useful?

a restart typically fixes the problem...

#ktistec #crystallang

heck yeah! too bad i need to support those older versions!

link: Checklist For Avoiding Or Fixing Query Planner Problems

#sqlite #ktistec

ktistec is fast but i'm thinking about making the responses to obvious vulnerability scans artificially sloooow... (opt-in, of course.)

#ktistec

When you optimize a Crystal program, pay attention to language features that inline code. For example, pay attention to how you use blocks (consequences here and here).

Also pay attention to how you use macros. Macros, like ECR.embed and Slang.embed, inline code at the point where they are invoked. This can be powerful, because macros actually generate code—but, ten invocations later, you have ten copies of the code.

Here's a case of too many copies, but with a very happy ending...

Ktistec uses both ECR.embed and Slang.embed to generate web pages from views and partials. I wrote code to count the number of places Ktistec used embed for each view and partial it renders. There's a long tail, but here are the big ones:

| src/views/layouts/default.html.ecr           | 204 |
| src/views/partials/modals.html.slang         | 204 |
| src/views/partials/header.html.slang         | 204 |
| src/views/partials/footer.html.slang         | 204 |
| src/views/pages/generic.html.slang           | 155 |
| src/views/partials/object/label.html.slang   |  36 |
| src/views/partials/object/content.html.slang |  36 |
| src/views/partials/collection.json.ecr       |  28 |
| src/views/partials/thread.html.slang         |  12 |
| src/views/partials/detail.html.slang         |  12 |
| src/views/partials/object.html.slang         |  12 |
| src/views/partials/actor-panel.html.slang    |  11 |
| src/views/partials/object.json.ecr           |  11 |
| src/views/partials/paginator.html.slang      |  11 |
| src/views/objects/thread.json.ecr            |   8 |
| src/views/partials/activity/label.html.slang |   6 |
| src/views/mentions/index.json.ecr            |   6 |
| src/views/remote_follows/index.json.ecr      |   6 |
| src/views/settings/settings.json.ecr         |   6 |
| src/views/tags/index.json.ecr                |   6 |
| src/views/activities/activity.json.ecr       |   5 |
| src/views/partials/editor.html.slang         |   5 |
| src/views/objects/object.json.ecr            |   5 |
| src/views/actors/remote.json.ecr             |   4 |
...

The layout is part of every page and is rendered with every view, so lots of copies. Every page has a header and a footer (and some default modal dialogs) so you get those, too. The generic view is a little less obvious. It's used to render pages for which there is no more specific view—typically pages served for 400 Bad Request or 401 Unauthorized. Objects (posts) are rendered in a variety of contexts, so it's no surprise label.html.slang and content.html.slang are popular.

ECR.embed and Slang.embed inline templates at the point where they are invoked, but beyond that they don't really customize the generated code—they just duplicate it. What we want  is one function for each view or partial, which wraps embed and returns JSON or HTML.

Those changes mostly occur in commits from 399287cf to 4b025f50. To say that they made a huge difference is a gross understatement. Executable size decreased by ~13%. Build time decreased by ~50%, and the memory required to build decreased by ~30%.

#ktistec #crystallang #optimization

I just released v2.0.0-11 of ktistec. This release picks up a few small improvements I wanted to get into v2.0.0.

• Allow the server owner to adjust log level by component (inbox processing, etc.).
• After trying to access an internal page when signed out, redirect the user back to the internal page after signing in.
• Handle updates from Mastodon servers when the signature can't be verified.
• Implement the remote interaction/authorize interaction protocol (an upgrade from remote follow/authorize follow). (see #91)
• Don't convert text into a hashtag or mention in links and code. (see #97)

#ktistec

I just released v2.0.0-10 of ktistec. I expect this to be the last pre-release before releasing v2.0.0.

As ironic as it sounds, the Fediverse doesn't feel very federated. ActivityPub, in particular, doesn't account for the real topology of the Fediverse—large groups of users clustered together on large server instances. (Or maybe it does, and this is a feature, not a bug!) Exchanges are largely actor to actor, and large servers create the illusion of "a Fediverse" by pooling their local actors' aggregate inbound and outbound activity.

The consequence of this is that running a single-user instance can feel lonely.

This release finally tries to address that. Hashtags and threads are the backbone of expressed interests and conversations in the Fediverse. Ktistec now lets you follow hashtags and threads, and will proactively (but gently) pull relevant content in to your server. Most of the changes in the last year revolve around making this work well.

The rest of the changes are less visible:

• Substantial reduction in build times and memory required to build.
• Substantial reduction in database size (if you care to shrink it) and query performance.
• Substantial reduction in the time it takes to run tests.
• Tons and tons of refactoring.

You can see all of the changes here.

(So that it's clear, I have a massive amount of respect for anyone who builds software and gives it away for free. None of the decisions I've made with Ktistec should be taken as personal criticism of anyone else in this space!)

#ktistec