Console #30

Protein Folding, Decentralized Twitter, and Flags

Jackson Kelley

Dec 06, 2020

Flagr is a feature flagging, A/B testing, and dynamic configuration micro-service.

language: Go, stars: 1392, watchers: 74, forks: 121, issues: 47

last commit: November 10, 2020, first commit: October 02, 2017

ripgrep-all

rga is a line-oriented search tool that allows you to look for a regex in a multitude of file types. rga wraps the awesome ripgrep, and enables it to search in pdf, docx, sqlite, jpg, movie subtitles (mkv, mp4), etc.

language: Rust, stars: 3081, watchers: 32, forks: 69, issues: 32

last commit: December 03, 2020, first commit: June 04, 2019

twtxt

twtxt is a self-hosted, Twitter-like decentralized micro-blogging platform.

language: Go, stars: 334, watchers: 12, forks: 28, issues: 24

last commit: December 04, 2020, first commit: July 18, 2020

https://twtxt.net/user/prologic

ProSPr

ProSPr is an open source implementation of AlphaFold.

language: Jupyter Notebook, stars: 195, watchers:12, forks:50, issues:10

last commit: June 17, 2020, first commit: October 30, 2019

oasis

Oasis is a small statically-linked Linux system.

language: Roff/C/Lua, stars: 873, watchers: 33, forks: 18, issues: 7

last commit: December 04, 2020, first commit: April 15, 2016

An Interview With James Mills of txtwt

What is your background?

I started to learn how to program when I was quite young, at around age ~11 or 12. I don't quite remember, but I do remember writing my first program in BASIC on some old 8086 and 80286 machines back in the day.
I come from a Software Engineering / Computer Science background and graduated from Griffith University in Brisbane of Australia back in ~2006.
I've had a fairly varied career being anything from a System Administrator, Network Administrator, to Web Developer, Application Developer, Infrastructure Engineer, DevOps Engineer and Production Engineer. I've worked all over the industry in numerous startups and large corporations, the largest of which was Facebook from ~2015-2018.
I've done quite a lot of Python over the years but these days I write most of the software you see today in Go. I'm also quite proficient in all things UNIX/Linux, Shell and all various manners of technology. Bit of a "jack of all trades" so to speak!

What's an opinion you have that most people don't agree with?

Oooh good question! I think the one opinion I hold rather strongly that some would (and have) disagreed with me on is "People are entitled to their own opinions". I believe very firmly in this, we all have an equal opportunity to express ourselves and our opinions regardless of it's correctness or an individuals background.

What is one app on your phone that you can’t live without that you think others should know about?

Oh that's easy! I could not live without two Apps. Signal, which I use to keep in touch with close friends and family. And Goryon which is our Twt.social Mobile App that I use to "socialise" with others on the Web!

If you could dictate that everyone in the world should read one book, what would it be?

I'm a bit biased, I haven't read many "popular" books as such. If I had to choose I'd say everyone should read the book "UNIX and the Linux Philosophy". I read it back in ~2015 and it was a great book on how creating things in the open is arguably better in many respects.

If I gave you $10 million to invest in one thing right now, where would you put it?

I would invest in Twt.social and use the money to build a social media platform that truly respects an individual's privacy and data ownership. I would use the money to help educate ordinary folk on the impertinence of owning their own data and being in control of the digital content that they create every single day.

What would your strategy be to enact this education?

We have plans next to create a "Kids" version of Twtxt where basically young children from the ages ~10-15 are able to participate in a social media platform that is safe from bullying, harassment, predators, etc. I won't go into details of "how" but let's just say that it will be a set of "supervised pods" and our target market are children (like my wife and i) with children that would not allow their children to use social media at all.
So to answer your question in a round-about way, we would educate the children.
Of course, we will try to educate adults too, but that is likely much harder and will involve expensive things like marketing and so on.

What are you currently learning?

Right now, I'm learning how to build Linux Containers the hard way and trying to reimplement some of the core features of Docker without all the extra things that you mostly don't really use or wouldn't really use.

What resources do you use to stay up to date on software engineering?

Mostly blogs and now various feeds through https://twtxt.net/

Any blogs in particular?

Yeah quite a few actually:
New Scientist
Slashdot
Unexplained Mysteries
Github Blog
CNCF
Docker Blog
Golang News

Why was twtxt started?

I didn't actually start it! Felix, who used to go by the name/handle of @buckket, did back in ~2016. He created this very simple file format / spec called twtxt. I just came across it in Aug 2020 and realised it had far more potential that what Felix originally envisioned, so, I set about building jointwt/twtxt, which powers pods like https://twtxt.net/ (my pod). This initially started out as just an ordinary Web App / Client but quickly turned into a Multi-User Web-app Client, then grew an API, then we built a Mobile App, a Javascript library, Bots, etc. What you see today is basically Twtter™ more-or-less but without the bloat, no advertising, no tracking of users, etc.

I noticed there were quite a few questions related to twttxt integrations with existing decentralized protocols when I posted about it (1,2,3,4). Would you like to respond to any of these questions?

Yeah I've seen the questions and honestly we (the development team) have thought long and hard about the benefits of integrating with the existing so-called "Fediverse" and exposing an ActivityPub compatible API. We believe it is better if we don't for a number of reasons. There is a good issue we went through on How does Twtxt compare to Mastodon where we spent some considerable time exploring this avenue. In the end we decided against integrating with Mastodon as it didn't really fit what we were trying to build. For example Mastodon and ActivityPub (the protocol used) supports things like "Reposts" and "Likes" which are things we believe should NOT be part of any social media free of bias.

Are there any overarching goals of twtxt that drive design or implementation, and, if so, what tradeoffs have been made in twtxt as a consequence of these goals?

Yes, quite a few actually, for example, we have very deliberately picked the smallest number of features that have the highest value to the user experience. We have ensured that the UI/UX is as simple to use as possible, even for the most non-technical mixed person (my old man even uses Twtxt!). We ensure that the components and libraries we pick are as small and lightweight as possible so as not to effect user experience in places where high-speed Internet is not as accessible. We also ensure that the backend is a highly optimised as possible and suitable to be run on very small devices, for example a Raspberry Pi.
These all drive the influence of the design and the overall user experience.

What is the most challenging problem that’s been solved in twtxt, so far?

Probably the most significant was a recent crash of the backend that occurred every day at 10am AEST (00:00 UTC) that was the result of the underlying database used to store user accounts, feeds and login details. You can read about it here and here.

What is your typical approach to debugging issues filed in the twtxt repo?

Data, data, data! I try to understand what's happening to the backend system be measuring it as best I can with tools like Memory Profilers in real-time and Prometheus metrics and using tools like Grafana.

Are these metrics built into the twtxt pods?

The backend has an integrated Prometheus client library that I use to "expose" key system/backend metrics, like, for example:
no. of anonymous sessions
no. of short login sessions
no. of database (long-lived) sessions
no. of feeds being fetched
memory, cpu, heap, gc time, etc
feed cache duration
no. of twts archived
errors
So, yeah, it is important that the backend is healthy and stable and highly optimized. I use this data to ensure this.

What is the release process like for twtxt?

The release process is meant to be a weekly to bi-weekly process. But this is a bit held up at the moment due to this issue -- We would greatly appreciate some help in this area so we can again provide prebuilt binaries more easily as part of our release process.

How do you intend to eventually monetize twtxt?

Twt.social is our plan to monetise what we've done here. Twtxt.net is just one Pod of hopefully many and just so happens to be the first inaugural pod (my pod). There are a few other pods that we fully managed and host on behalf of other pod owners today, and there is even one we don't manage that someone in the community spun up at twt.u53.net. Our intention is that we offer hosted, fully managed pods, that we sell as a service to pod owners wishing to form their own communities with the software and ecosystem we've created. Users would join these pods for free, so the cost of ownership is only on the pod owner. Our plan is to make this as inexpensive as possible at a price point of no more than $1.99 USD/month which we believe we can achieve (as long as we get enough uptake!).

What is the best way for a new developer to contribute to twtxt?

Please have a read of our README! It is all outlined there and we would LOVE every new contributor to the project! We have so much to refine and polish! Its pretty good now, but it would be a lot better!

If you plan to continue developing twtxt, where do you see the project heading next?

Next, we want to solidify more of the features that allow users and pods to discover each other more easily. We already have many intra-Pod features and discovery working quite nicely. We'd like to integrate more with the IndieWeb (we already integrate with WebMentions, and we plan to integrate with IndieAuth). We also want to revamp our "Feeds", so discovering and creating new custom feeds based on various sources is both easier and more discoverable. As a logical extension to this we'd also like to create a framework around creating "Automated Feeds" or "Bots".
In the long-run we also hope to attract business to Twt.social where we would "white label" and "brand" custom Twtxt pods to those businesses and charge money for this and/or host on behalf of small-medium businesses. An idea we had around this is that business could use this platform to either write short-form posts (Twts), long-form posts (Blogs), and potentially even promote their own goods and services. The upside is that since there is no data on users or their behaviour, users won't get annoyed with advertising being "shoved" in their face. The downside for businesses is that if they push their "agenda" a bit too hard users may decide to either "Unfollow" or "Mute" those feeds from those businesses. We think this is a good thing, it sort of turns advertising upside down!

Where do you see software development heading in the future?

I think software development will largely remain the same. Sure, things will be bigger, faster, better, and there will be heaps more data to be collected, analysed and stored. This last part is very important as we're heading into an era of "questionable privacy" and "data ownership". I am very worried that if we don't push enough for people to be aware of the trends it may be too late.