Console #85 -- Weird, Spacers, and CrowdSec
An Interview With Philippe and Thibault of CrowdSec
Sponsorship
If you, or someone you know, is interested in sponsoring the newsletter, please reach out at console.substack@gmail.com
Not subscribed to Console? Subscribe now to get a list of new open-source projects curated by an Amazon engineer in your email every week.
Already subscribed? Why not spread the word by forwarding Console to the best engineer you know?
Projects
crowdsec
CrowdSec is an open-source and participative IPS able to analyze visitor behavior & provide an adapted response to all kinds of attacks. It also leverages the crowd power to generate a global CTI database to protect a users network.
language: Go, stars: 4106, watchers: 89, forks: 196, issues: 78
last commit: December 06, 2021, first commit: May 15, 2020
social: https://twitter.com/Crowd_Security
repo: https://github.com/crowdsecurity/crowdsec
weird
weird is the next iteration of weir which was the next iteration of snek. The library is written to be useful for a broad range of ways in which I create art using generative algorithms.
language: Common Lisp, stars: 1103, watchers: 17, forks: 30, issues: 0
last commit: December 16, 2021, first commit: December 12, 2021
social: https://twitter.com/inconvergent
repo: https://github.com/inconvergent/weird
spacers
Spacers is a JavaScript library for adding spacers to an element for no-code tools.
language: JavaScript, stars: 186, watchers: 2, forks: 3, issues: 0
last commit: December 19, 2021, first commit: March 28, 2021
social: https://twitter.com/actuallyakash
repo: https://github.com/actuallyakash/spacers
The Console Career Service
Want to make more money for your work? Let us find you a new, higher-paying job for free! We’ve already landed 2 Console readers direct first-round interviews. Sign up for The Console Career Service today! The benefits of signing up include:
Automatic first-round interviews
One application, many jobs (1:N matching)
Free candidate preparation service
New opportunities updated regularly
All roles, from PM to SWE to BizOps
High potential, venture-backed, and open-source opportunities
Even if you’re not actively looking, why not let us see what’s out there for you?
Sign up for free in less than 5 minutes👇
An Interview With Thibault & Philippe of CrowdSec
Hey guys! Thanks for joining us. Philippe, let’s start with your background. Where have you worked in the past, where are you from, how did you learn how to program, what languages or frameworks do you like?
Being a French citizen, I graduated from a Paris-based IT engineering school called Epita, back in 1999, with a major in Network & Security. I created my 1st company right after my graduation, dealing initially with red team pentesting, but we later on extended our operations to ecommerce & high security hosting. It was sold in 2016.
I’m not really programming anymore and was never a real good programmer in the first place, but I tinker(ed) with C, Arduino C, Python, Bash, Perl and back in the past ASM, Pascal and few other outdated stuff. Nowadays, Python is still my goto language when I want to code something quickly.
Who or what are your biggest influences as a developer?
Linus Torvalds. One can say whatever they want about him, being harsh or brutal, he has set standards for the world operating system and made it the most robust piece of software ever.
What's an opinion you have that most people don't agree with?
There will be a hard crash of the Internet at some point. Someone will control so many mobile devices through a worm that he’ll have the on/off button on our digital life.
What’s your most controversial programming opinion?
Strictly typed languages aren’t that much better than loosely typed ones (to me it’s the compiler’s job).
What is your favorite software tool?
Visual editor.
If you could dictate that everyone in the world should read one book, what would it be?
The Alchemist, Paolo Coelho
If you had to suggest 1 person developers should follow, who would it be?
John Carmack still counts?
If you could teach every 12 year old in the world one thing, what would it be and why?
You can be anything, be bold, be curious, be brave, the world is in need of you.
If I gave you $10 million to invest in one thing right now, where would you put it?
CrowdSec ;-) Because we will make the Internet a safer place for everyone.
What in particular would you spend it on?
Mainly investing in the community, the support & the communication areas. The company would benefit from more visibility, by accelerating the adoption pace, hence the network effect, making it even more efficient in even less time.
If I gave you $100 million to invest in one thing right now, where would you put it?
One of the most promising fusion reactor startups. This is the future of mankind, if any.
Or SpaceX because they might be the future of mankind in space. It’s sad though, because no privately held company should be in position to dictate rules over such a key part of our future.
What are you currently learning?
Macroeconomics, Quantum chromo dynamic theories and being a father.
What resources are you using to learn these things?
I mainly quench my thirst for knowledge from quality news outlets, arxiv, podcasts. There are many channels which can help you catch up with missing knowledge. In math for example, I’m fond of 3 blue 1 brown on youtube. There are other videos where you can learn how to unpack Einstein’s field equations, MOOCs from prestigious universities to learn about Quantum physics. Did you know some of the lectures of Richard Feynman can be seen on Youtube?
As for fathering, it’s mainly field work :)
What have you been listening to lately?
How do you separate good project ideas from bad ones?
1/ Usefulness to others
2/ Economic viability or social positive impact
3/ Capacity to execute with existing resources &&|| network
4/ Fun & will to commit on long term
5/ Available time to commit to it
6/ Existing competition
7/ Time to market
8/ VCs appetite & understanding level of the subject
9/ Stock market momentum (if a crash is inbound, time your raise carefully)
What’s the funniest GitHub issue you’ve received?
Why was CrowdSec started?
Because it was validating the 9 points above.
Because we could do it and there is a real need for people to be able to defend themselves on the Internet, with free, efficient open source tools.
Where did the name for CrowdSec come from?
It actually stands for Crowd Security, to highlight the fact that we are leveraging the power / wisdom of the crowd to tackle the mass scale cyber criminality.
Who, or what was the biggest inspiration for CrowdSec?
Waze, for the “together” part of the project, the network effect. We were users and liked a lot the Fail2ban project, which was already identifying some behaviors in logs (mainly credential bruteforce) 16 years ago.
Are there any overarching goals of CrowdSec that drive design or implementation, and, if so, what trade-offs have been made in CrowdSec as a consequence of these goals?
Accessibility and lower technical barrier of use for the software is one of our targets to achieve our goal of the “biggest” detection network ever. To achieve this, we often do trade off for simplicity, sometimes at the expense of potential optimizations that we decide to not implement because it would introduce additional complexity to the users' experience.
What is the most challenging problem that’s been solved in CrowdSec, so far?
It is a difficult question to answer. I guess in terms of overall project, it would be to allow both complex and simple setups while hiding the complexity for simple users. While CrowdSec itself is a distributed software, most users with the simple setup can live completely unaware of this.
Are there any competitors or projects similar to CrowdSec? If so, what were they lacking that made you consider building something new?
There are CTI services, other IDS, IPS or IDPS, but none to my best knowledge combines our core pillars: Open source, free, participative, IDPS generating a worldscale CTI network.
What was the most surprising thing you learned while working on CrowdSec?
It’s surprisingly hard to convince people the product is free and that we are not trying to trick them. A lot of people seem to think we’re up to some evil, but actually the plain truth is a network of this size and quality is the only intrinsic value we need to go on with our company. It’s also probably because a lot of people oppose open source and monetization. They seem to think open source coders must be some sort of monks feeding on edible moss and little forest animals to survive, while giving away their highly demanded skills for free.
What is your typical approach to debugging issues filed in the CrowdSec repo?
We’re lucky enough to have easy reproducibility : live logs and cold logs can be processed alike. So, if a user files a bug about “crowdsec didn’t detect X” or “I got a false positive with Y”, it’s trivial to reproduce the bug with minimal effort. Alongside this, we capitalize on functional tests for both software and configurations and unit tests on the code to ensure high development velocity, so we limit “surprises”.
What is the release process like for CrowdSec?
Technically speaking, the process is heavily automated, as it is mostly about getting packages shipped: pre-releases (on GitHub) lead to packages being distributed into the testing repositories and “public” releases lead to packages being shipped to the main repositories.
From a cycle point of view, there is no hard rule here. We release when a significant feature is ready to be shipped, and by significant I do not only mean in terms of code, but sometimes in terms of community impact. For example, we just merged Docker datasource support, and it might lead to a minor release because people have been asking for it. Overall, we try to ship fast & early :)
How is CrowdSec currently monetized?
Yes absolutely. The free, open source product (IDS / IPS) will stay free forever. Users will also benefit from the sightings of the community they participate in. If you run an anti-credential bruteforce scenario on your machine, you get all the (curated) signals from all other users running the same scenario.
What’s monetized is the access to this IP database if you do not participate in the network (you do not share the signals), some premium features like 3rd party blocklist, or “Am I under attack?”, “Am I attacking the others?”, Forensic, Large structure management system (both on the human side & machine deployment), etc. All of those are SaaS based.
If it’s already monetized, what is your main source of revenue?
We have 3 different tiers:
SMB : 50€ / month
Enterprise : tailor made price
API: (for IoT devices for exemple, to be able to validate an IP before accepting a new login attempt for exemple) volume based pricing
It’s very early for us in terms of monetization but I’d bet the API one will be the largest income source.
How do you balance your work on open-source with your day job and other responsibilities?
By being a real company and paying people. Being open source doesn't mean making no money for us, as long as the community is benefiting from what we do for free. Corps can & will pay for the premium services we create around the product. So no need to arbitrate.
Do you think any of your projects do more harm than good?
Here I highly doubt that since we clearly and frontally oppose cybercriminals.
What is the best way for a new developer to contribute to CrowdSec?
If you plan to continue developing CrowdSec, where do you see the project heading next?
We definitely do commit for a long term adventure here so the project, hopefully, will one day gather signals from millions of machines and help the greatest number to defend themselves.
What motivates you to continue contributing to CrowdSec?
We have a mission, means to realise it and a fantastic team to collaborate with.
Are there any other projects besides CrowdSec that you’re working on?
No.
Do you have any other project ideas that you haven’t started?
194258
How about your top 2?
One I wanted to create at some point is a project I call “the admins”.
The point would be to create a system capable of creating a shadow version of any AWS environment. Then you take tickets level 1, 2 and 3 from clients, on the fly. A ticket level 1 would be paid $10, level 2 $100, level 3 $1000. And you propose those bounties to an extremely large crowd worldwide, a bit like 99 designs. Then hundreds of thousands of admins in the world can deal with your sysadmin tasks. Add a bit of certification & training before they can act and next thing you know, you have the biggest WW sysadmin force, with a follow the sun support as a side benefit. The client then marks the ticket as solved if he is satisfied, otherwise, the shadow servers/infra are destroyed, regenerated and the ticket is passed to the next sysadmin. If an admin has bad intentions, he can only kill a shadow environment.
Another one relates to Drones. I love building them and piloting them, but I feel it would be super fun to have a HUD in your goggles, displaying other players and being able to shoot them down (at least virtually). Also we could think of a trainer to learn how to make your stunts, that would validate them in your googles by showing you an overlay of arrows to follow and how sticks should be moved to achieve the desired trick.
And… so many more.
Where do you see software development heading next?
I’m of the opinion that a common universal syntax isn’t that far-fetched of a dream. At least for the fundamentals of every language (like variable assignment, loops, if statements, etc.) That would make language learning and switching so much easier. Obviously, stuff done in Rust and in Js aren’t the same, the goals of the languages are different in nature, but the fundamentals are common.
Where do you see open-source heading next?
It’s already on Mars, what do you want more?
Do you have any suggestions for someone trying to make their first contribution to an open-source project?
Connect with the project leaders on their favorite com channel, mingle with the team, exchange ideas, get some support from them and make your 1st PR. Open source is all about sharing and dialog so communication is key.
What was the hardest lesson you learned during the project lifecycle?
On my end, the hardest lesson was probably that no matter the amount of dedication or love you give to a feature, no matter how deeply you thought it through, now matter how long you work in an industry, it’s really hard to know what users really want on a large scale.
I mean this could sound simple, but if you are not Apple, with a capacity to interrogate 10000 users about what they would find useful in your next product iteration, you basically are left with educated guesses.
And you then implement a super-duper-cool feature, investing time and money in it (not talking about emotion), and … no one uses it.
A friend of mine, Yoav Kutner (former founder of Magento, CTO) told me one day about one of those features (it was about some ubiquitous cart that would follow you across devices/browsers to start somewhere your journey and end it on another machine). He thought it was so cool, based on his own experience, to be able to seamlessly give time to your shopping sessions here and there, when you can, from where you want. Well, close to no one used it and he was disappointed, but everyone wanted complex discount configuration systems so badly…
You should definitely make an interview with him, brilliant and opinionated, and muuuuch more of a dev than I will ever be.
As a lesson, I’d say release fast, do short cycles, make baby steps, while always remembering that a botched release might kill the whole project :) Especially when it comes to cybersecurity. Q&A people. Always.
So, release fast, but remember people are lazy and do not update software. Thus, hastily written code as a “technical compromise” to ship faster is going to be with you for *years*.