Spawnfest 2017!

Last weekend (9-10.12) I took part in SpawnFest2017 and I’ve got some thoughts to share

What is it?

SpawnFest is a hackaton (programming challenge to write a project in some limited time) that was all about BEAM – Erlang’s virtual machine. All projects have to run on top of that (so you had to write it in Erlang/Elixir/LFE or something other from this family). You could add JS for frontend, use some third-party libs, etc. This was first edition in 5 years, so I had to take a part as I’m big BEAM enthusiast.

Warning, personal stuff below!

It was my firs (organised) hackaton. 48h deadline was quite stressful, and a lot of things didn’t work. Being at beginner+ level didn’t help either. But it was so much fun! I worked with my 3 friends on silly idea that we managed to bring to life. I learned a lot about not only Elixir, but developing and working under that kind of a stress. Memes were created and we never lost good spirit. A lot of code created was poor quality, but it didn’t matter. We did something in language we’re excited about, together, in less than 48 hours. That was magical. I cannot recommend enough taking part in SpawnFest – it was well organised, everyone was welcome and all the good things I cannot describe with words. It felt really awesome! Funny thing, when this all was over I was reminded that there are winners and prizes (Judges are still voting), but for me I already got the best prize – which is experience from this event

So, what did we do?

Application to monitor plants. We planned to have thermometer and proximity sensor, but we didn’t manage to get it work in time, so we added buzzer to hydration and humidity sensors. This app (written using Nerves) communicated from Raspberry Pi to our webapp written in Elixir, that was deployed in Heroku. What was shocking – how damn fast it was! Below I’ll paste README description of short video demo that we made.

There’s Elixir app with Phoenix fronend opened, showing sensor output. Sensors are connected to raspberry pi. When sensors are dry it will show cactus, when wet it will show water drop. First there’s humidity sensor – if we spray it, the second image will change. As it was not dried properly you can see some changes later, as water drops flows down the sensor. Next there’s hydration sensor put in the glass of water – first image will change. Below the images are charts with sensors data grouped by hour.

Finally “warning” button is pressed, and buzzer turns on. “Warning” button is a switch, so pressing it again turns the buzzer off.

VIDEO

See you next year! 🙂

Today I Learned #2

While a back ago I did a little test. I read the Deliberate Vim book, did the exercises and decided to go full Vim. So I installed ViEmu to my VisualStudio 2015. Aaaand had a few struggles. Some shortcuts conflicts that I had to solve manually and still it wasn’t so convenient to use.

I ended up skipping ViEmu for VS2017. But it didn’t last for long – one day I noticed I’m really used to some of the Vim commands and it’s more difficult to work now without them. So I did some research, and got a great recipe!

If you have anything keyboard-changing installed already (like Resharper) – reset all shortcuts to default – so you get only VisualStudio’s default key bindings. After that install ViEmu and let it take all the shortcuts it needs. Finally, install Resharper/apply Resharper scheme. Those are the steps that will provide minimal friction while working with Vim in VS.

 

Bonus round: For VisualStudio Code – just install a plugin, it works great!

PureScript: The First Look

After ElixirConfEu I decided to try PureScript. Partly for yet another frontend try, party because it looked interesting and partly because I wanted a little break with something way different and new.

I read a bit of this awesome book and did some of the exercises. Here are some first thoughts:

The bad:

  • Install npm, to install bower, to install dependencies. I get it, it’s JS. It’s frontend. It’s too young to have its own package manager and it’s even better that it uses a common tool. But come one, you can do it better than this. Especially if apparently you can compile to C/Erlang/someothers – not only JS.
  • Haskell-like docs. It’s not the worst, but it’s really not newbie friendly.

The good:

  • Types! This is the most awesome thing, really. You specify what type goes in, what goes out. It’s marvelous, especially for someone with a strong C# background.
  • Quite obvious, FP approach
  • Book (mentioned earlier) is a great learning resource. It’s free and written by PS creator
  • I know that it’s not something crucial, but I really like the syntax
  • How easy is to start, VSCode tools are great, you can google stuff and get some answers already
  • The community seems small but nice
  • Error messages are really, really helpful!

As you can see, there are way more “goods” than “bads”. Should you try it? Definitely? Should you use it in your pet project? Sure! Should you use it in production? It depends 😉 After going with dotnet core RC1 in production I’d say “hell yeah” but this requires the team that wants (not “can“; want!) to handle it, so my answer here is “it depends”. Nevertheless, I’m hyped and will do something more with it, but the break is over and I’m heading back to Beam world now.

Today I Learned #1

While using EntityFramework in my integration tests (which is a separate topic 😉 ) I discovered quite interesting thing. I guess this may be obvious to some, but I learned Entity “the hard way” jumping into an app with Entity already in place and had to adapt – this was my first app with a database by the way.

So if you add entities to your context I’m used to adding all entities to context, so the code would look like

using (var ctx = new Context())
{
    var first = new FirstEntity { .. };
    var second = new SecondEntity { .. };

    ctx.FirstEntities.Add(first);
    ctx.SecondEntities.Add(second);
    ctx.SaveChanges();
}

But if entities are related, you can safely do this

using (var ctx = new Context())
{
    var first = new FirstEntity { .. };
    var second = new SecondEntity { Relation = first };

    //this will also take care of the first one!
    ctx.SecondEntities.Add(second); 
    ctx.SaveChanges();
}

Or even this!

using (var ctx = new Context())
{
    var second = new SecondEntity { Relation = new FirstEntity{ .. } };

    ctx.SecondEntities.Add(second);
    ctx.SaveChanges();
}

It’s nice and saves some typing! 🙂

Property testing

During ElixirConfEu in Barcelona, I learned about Property Testing. It looks pretty neat and it got me interested. Basics sound quite easy but there’s more than meets the eye and I’ve been reading/listening about it for a while.
As I don’t feel comfortable enough to do a deep dive into the topic I will do an introduction to it. After I get a deeper understanding with some “real life” examples (or maybe doing them myself) I will write a follow-up.

Property testing is a term originating from Haskell lib called QuickCheck. It was created to ease the pain of writing many tests. Instead of writing n specific unit test you can generate them.

Using QuickCheck (here is the list of ports to your language of choice) you define a property of a piece of code you’re testing.

For trivial example – if you were to write your own ordering function you can define few properties – if you order it twice the result won’t change, the only change is the position of elements (so you don’t hanger values) and so on.

QuickCheck then generates data, runs n tests using this random data and if it finds failing case it executes something called shrinking – trying to find minimal failing case. It can ease up debugging or seeing straight away what’s wrong.

While it’s all fun, I’m still not sure what are the cases in a commercial code where this is the best approach. Also, turns out that properties also form kind of patterns – and I’m yet to learn about all this.

Nevertheless, I’m quite hyped and want to learn more – it seems more of easy to get, hard to master useful tool than a novelty, but only time will tell.

OpenSettlersII #6

As always, a commit!

Slowly crawling towards functioning lib.

This commit is not really that different from other, yet I started using Elixir convetion for return values: the tuple {:ok, response}/{:error, message}. I am quite not happy with  the as_server function and I guess I will rename/rewrite it later. It’s quite obvious what it does – checks the rightmost bit and responds and sets “as server” to according boolean value.

Also, encoding and decoding version – I have no way of validating it, but I’m not sure if I should. This is one of two things I need to check before proceeding – another one being the Signature – I’m not sure if it’s version specific or not, so that’s something to get to know before coding it.

And that’s it for now 🙂

Elixir Conf Eu

Hello!

There was a week without updates as I went to the Barcelona to attent Elixir Conf Eu – and afterwards made a short vacation. This time I won’t be as specific as I was with ErlangFactoryLite; it was way bigger conference and I didn’t get to see all talks as well. Truth be told – but it’s a problem with any bigger conference – overall level of talks was lower than EFL’s ones – but it’s inevitable. Organisation was really good, as the venue.

I have some stuff to share, less now, more later when I deep dive into them.

First, there’s something called QuickCheck. It’s enables to do Property Testing – I’m not feeling comfortable enough with my knowledge of this subject to try explaning it, but I’m looking forward getting to know it more and write about it here!

Another thing is PureScript (looks like you can read a recommended book here for free!). It’s compiled to JavaScript (as all of those new front-end stuff), but apparently you can also compile to Erlang and some other as well! It’s influenced by Haskell, more elastic and powerful than Elm (which is easy to learn, but apparently if your app doesn’t fit Elm’s architecture you will have a lot of pain). I never did anything serious in frontend from start to finish (I did some work with React + Cerebral, but I wouldn’t call myself front-end specialist) so I guess this could be an interesting start.

Nerves strikes again, as this time distributed computing on Rasps was shown. It’s nice to see that this project grows. And also: nice project to get started: make temperature controller for beer fermentation (old fridge, old lightbulb for hear and rasp pi running nerves)!

 

OpenSettlers#5

As always – a commit!

Here’s a thing I struggled while doing previous commit – decoding command to something more structurized – according to docs it’s composed of command name and body – all coming in one binary. I had to get the name and body (if any)

It seems like you have to use ::binary-size for the first one, and just ::binary for the rest – I may be wrong thou and I will have to research that more in the future.

It’s slowly starting to be a mess – that’s why I refactored enconding a bit to make a use of pattern matching. Not it’s more “elixir style”. Still I feel the lack of the bigger picture – I guess rushing into it wasn’t as good idea as first though 😉 I will have to do some more research and maybe a post on a protocol – it will surely help me. For example – turns out that command will never have “more” frames incoming – as you can see in changes in code – now all command have “more” set as false (with is quite redundant).

Erlang Factory Lite Rome 2017 #1

Hello for the second (and last) part of EFL Rome2017 post! You can find previous one here

Music and Message Passing Concurrency by Joe Armstrong

Here goes the big name, Joe himself. While it may suggest a lot about music, it was more about messaging and integration. Joe showed how he could remotely control Sonic Pi. He was glad that there’s an other way to control a program “than clicking a bloody mouse!”.

He did that sending properly coded messages to UDP port. And here it all started. Joe is a big supporter of messaging based communication. API suposes a programming language, so it’s not a best way – you’re tied to programming language. As the main topic emerged once again – how to integrate and there are some programming languages better to do other things – so you’d naturally like to use them for specific purposes and not be stuck with one.

You should also treat specific parts/modules/apps like a black boxes – you know nothing (even the language used) except for how to communicate (using messages, of course 😉 ) – “the protocol only matters.”

  1. Pick transport (TCP/UDP)
  2. Pick encoding (XML/JSON/YAML/etc)
  3. Pick protocol description (RFC/UBF/etc)
Some combinations for you to pick from
With the Joe himself!

Joe picked OSC over TCP/UPD with some English to describe it. OSC is a very simple encoding – and it has “simplicity by design”, as Joe said, “if you can’t create complex data structures, the interface will be simple and easy to understand”.

“if you can’t create complex data structures, the interface will be simple and easy to understand”

Which is another way worth remebering. Complicated systems are easy – you just keep adding stuff. Simple systems are difficult to make.

I think thou – if one thing should be remembered from this talk it’s fact, that we should be able to understand the system by looking on the messages going in and out.

Adopting Elixir in a 10 Year Old Codebase by Michael Klishin

While it seems like adding Elixir to some Ruby ecosystem, it was actually using Elixir in Erlang project. They did a CLI tool in Elixir as a “check” with ~8k LOC, ~750 tests and 70 CL commands available.

I will just point out good:

  1. It’s more approachable than Erlang
  2. It’s a recriutment honeypot – easier to lure people into project
  3. Has decent standard lib
  4. Potentian contributors could have not contributed if it was in Erlang

And bad things he learned:

  1. Integration with Erlang.mk is a pain
  2. Elixir/Erlang data type mismatches
  3. String vs Binary
  4. Some Elixir libs are a one man show ( 😉 )

Overall Michael feels like Elixir was worth adopting as sees a bright future for it – and you can utilize it even without agents, macros and sweet libs. He was even more optimistic for Elixir than for Erlang itself.

A Little Replica of the Internet in Elixir by Ju Liu

He describes himself as a Mad Scientist – and, boy, he is!

Basically it was a little trivia on the Internet and his attempt to do a small replica of it’s routing system using Elixir (and nerves). He had 2 RP3s connected, simulating North America and Europe with routing links. There’s a bit of how the Internet works and how it connects – he used that common knowledge to simulate it and it worked like a charm. There a code online if you want to look at it!

This is the Internet! Be careful not to break it!

The real show started when he took the router, connected third Rasp and added Asia – it all worked! It was a nice show for the Nerves project to show what you can do with them. Funny thing – the most problems, and the slowest part was HTML+JS frontend where all the arrows where hacked as separate CSS elements – so it crashed when connection number rose. Still, great project and you can look at it on github!

Embrace the Database with Ecto by Josh Branchaud

Basically it was how to use the Ecto and how to query the database. I really don’t know if it was the high temperature in the room, or it was just quite obvious to me – but I cannot say that I didn’t find anything at all – there was one statement that I won’t say if I find true or false – because it’s strengh is that it provokes some interesting conversations:

“The database should be the ultimate gatekeeper – it should validate the data coming in”

Monitoring and Pre-emptive support: The road to five nines on the Beam by Francesco Cesarini

It was a great talk with a lot of focusing on “no single point of failure” “we need at least 2 of everything”. He showed 2 types of monitoring usefulness – you can prevent failures, or do quick post-mortems. With monitoring you can prove your innocence in 1.5 minutes – without trying to replicate the bug and trying to debug. While there’s a lot to be said – he said so many cool stories I guess you have to go and give it a listen yourself – this one seems almost identical.

And this was it – it was a great experience and possibility to talk with those people was sometimes eye-opening.

OpenSettlers#4

As always – commit!

I really need to spend more time. Today I struggled a lot with bitstrings. And still didn’t find an answer for what I was looking for, but I created a map for command without redundant info. So no more flags, only command name, data and size.

After push thou I saw a better solution – I could do pattern match on functions, so I could eliminate switch. I get a lot of mess with all those maps. I think that rethinking/refactoring it should happen sooner than I initially thought.

Also “size” field seems a bit off. Additionally I don’t feel as comfortable enough with specification as I was expecting, so I guess I’ll spend some more time with it – this should also get me up to speed.