Console 53

Hotdogs, Humbug, and Delta

Sponsorships

Streamr Network

The Streamr Network is an open-source, massively scalable pub/sub network for transporting encrypted data points to thousands of subscribers. You can use the network in your own applications via the SDKs or by directly calling the API. If you're an app builder, the Streamr Data Fund exists to support projects that further the vision of a decentralized data economy. All proposals are welcome at https://streamr.network/fund. Learn more about the Streamr Project at streamr.network.


delta

delta is a viewer for git and diff output.

language: Rust, stars: 8964, watchers: 43, forks: 139, issues: 104

last commit: May 10, 2021, first commit: June 23, 201

https://twitter.com/dandavison7

thdwb

The Hot Dog Web Browser is a web browser with its own layout and rendering engine, parsers, and UI toolkit, made from scratch entirely in Go.

language: Go, stars: 841, watchers: 17, forks: 39, issues: 4

last commit: May 10, 2021, first commit: July 29, 2019

https://twitter.com/danfragoso

humbug

Humbug helps you understand what keeps users coming back to your developer tool, as well as any friction they experience.

language: Python, stars: 12, watchers: 4, forks: 5, issues: 11

last commit: April 30, 2021, first commit: February 24, 2021

https://twitter.com/Bugout_dev


An Interview With Neeraj Kashyap of Bugout.dev

Hey Neeraj! Thanks for joining us! Let’s start with your background. Where have you worked in the past, where are you from, how did you learn how to program, what languages or frameworks do you like, etc?

One of my early programs drew geometrical figures for my math homework (hundreds of problems) that I was too lazy to draw by hand. This was a QBasic program, which I learned by reading the Absolute Beginner’s Guide to Qbasic, and reading the source code of Gorillas.

I have had two great loves in programming:
Common Lisp - Was introduced to Lisp in an AI class in college, and immediately fell in love. I spent months worth of late nights writing a computer algebra system in Lisp.
Python - Was introduced to Python when I was doing some number theoretic experiments as part of my PhD work, in 2012. Fell in love with the expressiveness of the language, and have been obsessed ever since.

I started my life as a mathematician and initially had no desire to do anything but teach mathematics and do mathematical research. That changed when my grandmother was diagnosed with progressive supranuclear palsy. Watching her progression with the disease made me realize that I wanted to build things that were of actual use to people - people like her, and like my mother who was her primary caregiver.

I spent two years in Japan as a postdoctoral researcher, developing an algorithm to help diagnose neurodegenerative disorders earlier than they are normally caught.

Progress is slow in academia, so I took a chance and moved to the Bay Area to start a company, which didn’t work out. My journey since then has taken me to a couple of startups, and to Google, where I worked on Tensorflow and Cloud AI.

I am now working on Bugout. We are building tools which help creators of developer tools (libraries, command line tools, and APIs) understand who their users are, and how they are using their code.

Who or what are your biggest influences as a developer?

About 10 years ago, I took a course on recursion theory with Larry Moss, a logician at Indiana University. That course really made me fall in love with computing.

In that course, we would write programs against a register machine in a language called 1# in which the only symbols were “1” and “#”. One week, our assignment was to write a universal program - basically a 1# interpreter. Completing that project gave me the confidence to tackle other difficult problems in computer science.

What's an opinion you have that most people don't agree with?

Libraries and frameworks should be open source. Server code should not be open source.

If you run a server, that server’s users are trusting you with their data and with data about how they use your server. This is a serious responsibility. Besides other security measures, you should also rely on obfuscation. To whatever extent possible, limit the number of people who know how your server is implemented.

What’s your most controversial programming opinion?

bash is an amazing programming language. It gives you direct access to the wisdom of the ancients. You will be very well rewarded for writing at least one serious bash script a week.

What is one app on your phone that you can’t live without that you think others should know about?

tiktok :)

If you could dictate that everyone in the world should read one book, what would it be?

I would never dictate something like this. In fact, I have a counter recommendation - go out there and get your hands on some real garbage. I mean books that no one has ever recommended to anybody else in the history of humanity. Books that you know from the start that you are going to disagree with and vehemently. Books that nobody has even taken the trouble to read before. Read them.

If you are caught in an echo chamber, this is the surest way to break out of it.

You can find books like this at your public library and at library book sales. You can also find them on Amazon - just search for books that cost less than a dollar.

If you could teach every 12 year old in the world one thing, what would it be and why?

Don’t do your homework. It is a complete waste of your time. Create something instead.

If I gave you $10 million to invest in one thing right now, where would you put it?

I would set aside $8,000,000 in a trust for my family - children, wife, parents, sister, in-laws. Just to get rid of all financial worries.

I would invest the remaining $2,000,000 into Bugout.

What would you spend it on at Bugout?

We are currently hiring fairly technical people to join our marketing and sales teams. 1/4 of the resources would go there.

For us, developer relations and customer success are closely tied together. Improving our on-boarding and our documentation is becoming increasingly important. As is communicating the value of Bugout to the world, through blog posts, videos, etc. Our customer success team needs to be highly technical, as well - we intend to hire engineers into these roles. I would put 1/3 of the money there.

The rest would go towards hiring engineers, paying for infrastructure, and other assorted costs of running a business.

What are you currently learning?

Learning about different blockchain implementations.

Any blockchains in particular that you’re interested in?

I really like Ethereum. Polkadot is pretty interesting.

Don't understand Monero in detail yet, but it seems worth spending some time on.

What resources do you use to stay up to date on software engineering?

Curiosity. :)

I read Hackers News a lot, and also various subreddits for programming topics I’m interested in (e.g. r/programming, r/machinelearning).

What have you been listening to lately?

The last thing I listened to was the album Blade of the Ronin by Cannibal Ox.

How do you separate good project ideas from bad ones?

Are you still excited about an idea a week after you had it? If so, it’s a good idea. If not, move on with your life.

Why was Bugout started?

My co-founder (Sophia) and I really admire people who make things. Bugout embodies everything we think will be useful to the next generation of programmers.

How did you and Sophia meet?

We met at a healthcare startup called doc.ai. I was managing the data science team there and Sophia was head of communications.

Are there any overarching goals of Bugout that drive design or implementation?

Our immediate focus is on helping people who maintain libraries, APIs, or command line tools understand their users better.

They use Bugout to learn who their users are, how they use their software, and what kind of issues they experience. This knowledge comes in the form of usage metrics and crash reports. Our reporting libraries are freely available at https://github.com/bugout-dev/humbug.

Two key principles that drive our design and implementation:
1. Providing a magical user experience requires deep understanding of your users - something you can only get from the client side.
2. Privacy is paramount. Bugout does not transmit any data from any client without the user’s explicit consent.

What trade-offs have been made in Bugout as a consequence of these goals?

Not a technical trade-off, but a product trade-off.

Our end user’s privacy is our highest priority. We have added a mechanism that anyone can activate to disable all Bugout reporting in any tool that they use. They simply have to set the environment variable BUGGER_OFF=true and no Bugout reports will be sent back from their system for any tool that they use.

The trade-off here is that this short circuit makes Bugout a little bit less useful to customers since we report data about fewer users. But it is the only ethical choice.

What is the most challenging problem that’s been solved in Bugout, so far?

As part of our machine learning work, we need static analyzers that do more than understand code. We need analyzers that understand how code changes over time.

Since we couldn’t find such a project, we had to build our own - Locust. It produces a JSON object describing the changes to a code base over time, with time measured in git commits.

Interesting section: For the most part, we are doing standard abstract syntax tree traversals. The interesting part is the function which superimposes git diffs onto the results of these traversals (link).

We will be launching features in the coming month or two which use Locust data to drive code modifications.

Are there any competitors or projects similar to Bugout? If so, what were they lacking that made you consider building something new?

Our biggest competitor is Sentry.

If you want to use a tool like Sentry to collect usage or crash reports from your users’ machines, you have to implement all privacy-related functionality yourself - processing and storing user consent, serving GDPR requests for data access or data deletion.

Not only is it hard to get this right, but it is also a distraction from just building your product. We built Bugout to solve this pain.

What is your typical approach to debugging issues filed in Bugout repos?

We prefer that our users report issues to us on our community Slack. We are still at a scale where that’s more effective than working with issues on GitHub.

It’s easier for users to report problems or make requests because things are more casual on Slack.

It’s also better because we can interact in real time. Conversations on GitHub issues are asynchronous and it can sometimes take days or weeks to resolve simple matters.

Regardless of whether an issue is filed on GitHub or mentioned to us on Slack, our approach is similar. First we identify if it’s a bug, feature request, or a misunderstanding.

We try to resolve bugs within a day of being filed.

For feature requests or misunderstandings, we first talk to the reporter to really understand their use case.

If it’s something that Bugout is already capable of, we explain it to the reporter and also improve our documentation.

If it’s something new, we prioritize internally, tell the reporter when we think we will have bandwidth to work on it, and invite them to collaborate with us on it.

What is the release process like for Bugout?

We release several times a day in the repos that are seeing active development. All releases take place automatically upon merging into the main branches of our repositories.

Our APIs run purely on VMs on AWS. We use just plain old SSH to deploy. Here is an example of one of our deployment scripts.

Actually, we are pretty opinionated about deployments. I wrote a blog post about it at the end of last year: https://blog.bugout.dev/2020/12/03/break-out-of-the-deployment-death-loop/

How is Bugout monetized?

Our customers use the Bugout API to collect reports from their clients. We charge them per 1000 reports they post to our API.

What is the best way for a new developer to contribute to Bugout?

We have a few open source projects: https://github.com/bugout-dev

If any of them seem interesting to you, join us on the Bugout slack and ping me - my username is @zomglings. We can talk about what you would like to achieve, how that fits into the project in question, and I can onboard you onto the code base.

After that it’s Slack conversations, GitHub issues, and pull requests all the way.

If you plan to continue developing Bugout, where do you see the project heading next?

Our dream is that, when a bug report comes into a repo, Bugout can analyze what the issue is, modify the code base to fix the bug, and submit a pull request. This would give any programmer working on the issue a good place to start (and hopefully finish).

In the next couple of months we will start releasing some of our code modification features. We want to make it easy even for non-technical people to manage Bugout reporting in projects that they participate in.

We are also working on tighter integrations into developer environments. We just launched our VSCode extension in which programmers can hover over exceptions to see how many of their users have experienced that error recently, and even see the individual stack traces. This is something we will be expanding on.

Where do you see open-source heading next?

Regular contributors get the short end of the stick when it comes to sharing in the rewards for building a great project. This has led to a polarization in open source projects - projects that don’t have corporate backing tend to have very few regular contributors.

It takes time to build something really useful. A good team gets there faster. I hope that we can find more equitable ways of rewarding open source contributors so that we see more such teams.

There is a lot of potential for this kind of equity on public blockchains like Ethereum. You can tell from transactions how frequently particular methods were called on a contract. You could hold a percentage of the funds that were sent to the contract as a pool from which to pay out developers who had contributed to the contract’s development. The payout could be proportional to the amount of money spent on methods they contributed to.

A big problem is the problem of quantification. Is a one line bugfix worth more than a documentation change? How much more? How do we encode this difference in value into a program?

If last year’s Hacktoberfest drama is anything to go by, we will be contending with this issue very soon. This will be huge for open source, and for all software development.

Do you have any suggestions for someone trying to make their first contribution to an open-source project?

First, make sure your contribution is welcome.

The last thing you likely want is to spend your time building something only to have it summarily rejected because it doesn’t align with the project maintainers’ vision for the project. For most projects, it should be easy to get hold of the relevant maintainer and run your idea by them first - on a GitHub issue or over email or over Slack/Discord.

Next, make it easy for a reviewer to approve your change.

Open source reviewers are a busy lot. The more time they have to spend understanding and testing your change, the less likely it is to get merged. Document everything that you have done. If you made any decisions that weren’t obvious, explain your rationale. At the very least, run the existing tests and make sure they pass. Lint your code according to the project’s style guide (if one exists). If possible, add new tests for your change. Such a change would be a pleasure to review.

Finally, a piece of advice from a writing class I once took: When someone critiques your work, shut up and fucking listen!


Like what you saw here? Why not share it!

Share

Or, better yet, share Console!

Share Console

Also, don’t forget to subscribe to get a list of new open-source projects curated by an Amazon software engineer directly in your email every week.