Down from the tower.

OK. That's one time too many. Or even, quite too many times too many.

My beloved Eiffel compiler is really not production ready. It is the right tool to test ideas, but it is too brittle. The compiler was a very good one 15 years ago; but it did not evolve and is now quite behind the times.

Hacking on that compiler cannot stay a priority anymore. Let's face it, I am not up to the challenge. I feel like Atlas, trying to carry the thing on my frail shoulders.

Each time I try to make pwd evolve, I need to fix the compiler, or some standard library.

That needs far too much work and my patience is quite over.

I need my password manager. And I need it now. I want to use the man power I own (i.e. one our two hours per day, max) to make the best password manager ever. And no, I don't want to use a standard one, at least because it is the software I want to completely control.

So I will start anew.

My needs:

• Using standard technologies, so that others can contribute. I will use the lingua franca of free software: C.
• Using as many standard libraries as possible.
• Using an event-driven model, because I want to avoid threading issues. Not decided yet: zmq, libuv?
• Whenever relevant, build reusable libraries. That begins with libcad and yacjp.
• Good security, especially cryptography-wise. I still want to be able to exchange passwords between my server and a web client. I need layers of security above https which is the bare minimum.

I also need a name. Ideas are welcome :-)

I think I will not write again in this blog before next year. So merry Christmas to all, have a good time with your loved ones, and happy hacking!

PS– That does not mean that I completely forsake Liberty; I will gladly help people whenever such help is wanted. But hey! Judging from the current mail flow, it should not take a lot of my time. And that's an understatement.

Key properties

Hard times seem to be behind. Cross fingers.

Now that pwd works with a new format (JSON), I can start to implement a few enhancements I wanted to do.

The first is a bunch of key "properties": attach extra information to each key. For the moment, three properties are implemented: username, url, and tags.

I am quite proud of the tags system. It was really missing. It will help segregate keys on user criteria.

This is not yet over. I need more work on accessing and changing the key properties in the console and maybe the menu clients, and in the webclient.

MOre happy hacking to come.

Bug in the compiler.

JSON is ready in pwd. It should work. It will work.

Liberty Eiffel has a bug in the implementation of a quite brand new feature: the keyword "then" can now be used at the end of a function definition to assign an expression to Result, to be returned to the function caller. It helps writing short functions.

See Eiffel as an expression language.

The management of this expression is buggy; I am fixing it.

More next week.

Garbage collection

In Liberty Eiffel, when using the SmartEiffel garbage collector, a last collection is performed just before exit.

The standard Liberty Eiffel library uses the "dispose" method of disposable objects for contract checking; in particular, to check that resources that should be closed are closed.

Looks good? Yeah.

Except.

Now with mocks, contracts have to be checked too. But the latest garbage collection happens out of any scenario. It happens, well… Later; when the program is about to exit, when the test program has checked that every method that should have been called was indeed called, and so on. The test is over, we may safely exit.

And then, comes the GC, that calls dispose, that checks contracts, that calls… mocked methods! Weren't we finished?

As you can imagine, that does not end well. When not guided by a scenario, mocks don't know how to properly answer to a query.

Only one solution then: when using mock objects, that last GC should not happen, and that is that. After all, the SmartEiffel GC is the only GC I know that performs a GC on exit. BDW does not allow it.

So now the Liberty Eiffel mock framework disables the GC on exit.

Lo and behold, after almost a month, pwd tests pass again!

Happy coding indeed.

Design by Contract

Design by Contract is a fundamental aspect of the Eiffel language.

Methods have pre- and post-conditions; classes have invariants. Those contracts are checked even for heir classes.

This has a very important impact: mocks have contracts too.

Contract checking may be partially or completely removed; this way, Liberty Eiffel is able to generate executables with very high performance.

But when testing, one wants to check all modes. With, and without contracts.

So the tests must be properly equipped. That's what I learned this week: when using mocks in Eiffel, don't forget to bring answers to contract methods.

Mocks

This week is about mocks.

A few years ago I wrote a "mocker" tool for Liberty Eiffel.

In the case of pwd I need to mock actual classes from the library, where deferred methods (i.e. abstract features) are defined in one or more parent classes, with some implemented at different levels of the tree, and so on.

The "mocker" tool, with its simplistic approach (purely syntactical), was not up to the task.

So be it: I needed to write a new tool. One that could handle the whole semantics of an Eiffel class.

Liberty Eiffel already has similar tools; in particular, the "short" tool allows to display the complete interface of a class.

Lo and behold, the mock tool, now based on the Liberty Eiffel engine. It took me the whole week to write it.

But now that it works, I can come back to pwd and further my webclient tests.

The target is still acquired. Stay tuned.

Is it a duck?

Last week, I was so happy. I had a working webclient for pwd! My integration test script allowed me to start a web server (using lighttpd).

Enters Travis CI. This useful tool reminded me that integration tests are not enough. I had broken my unit tests. Not good!

OK, let's fix that. I thought it would be fast. It was not (indeed, it is not over).

I needed to work quite a lot this week. Eiffel allows "expanded classes"; that is, value object classes. That is very useful but almost as easily testable as singletons.

I kept those classes… as simple façades. The release code automatically wires the façade with its underlying implementation. The test code, on the other hand, allows to set a test implementation; in other words, a mock.

Of course, faster said than done. There were two bugs in Liberty Eiffel; I had to fix them.

I also worked a lot on Liberty's mocker tool and library to give it new possibilities.

That done, I began by generating mocks and preparing my scenarii. I also split the tests for better legibility.

The work is still ongoing.

My last words are for Eiffel afficionados, based on my freshest experience.

I had lost the idea that Liberty Eiffel's insert keyword (equivalent to ECMA's ugly inherit {NONE}), brought a whole interface to the class. Not what I wanted.

So, my advice: you should not insert a class if you don’t want to expose its features in your interface. Use an attribute instead. Using export {} is an alternate solution, albeit not pretty because you'd need to also insert ANY to keep valid export clauses for standard features.

If it’s not a duck, it should not quack.

Happy hacking!

Web access

At last the web client implementation of pwd works!

It means that very soon I will be able to query my passwords database using web access. (Of course, the zone is a private SSL zone with credentials access).

Next steps: a bit of HTML prettiness, and more features as announced previously.

Moods and features

One month without posting.

One month during which I quit my previous job, moved near to Paris, and started something new.

One month during which I had to think a bit about my open-source future.

I played with the idea of stepping away from Eiffel. Well, I cannot. Eiffel, and particularly Liberty Eiffel, is in my blood.

On the other hand, I really need new features for pwd.

The feature #1 one being a secure web access, to avoid having to resort to ssh from the office, while still being able to get my passwords.

The infrastructure is already there. I have my own server, with an nginx server, and a pam-protected private area (URL not disclosed, obviously).

The feature #2 is the ability to add data to each key:

• user name
• url
• tags

Compounded with this feature, I will need to change the internal format to something more flexible. Certainly JSON.

So that's my new aim. Enhancing pwd. And go on hacking, anyhow.

I guess I am back online.

New home address

Cadrature du code is changing its home URL. It is still hosted by Blogger but it is now accessible using my own domain name.

So from now on, this blog should be accessed through http://cadrature.cadrian.net

Of cource, Blogger's address stays available.

Gate's channels

This blog entry is not really structured. I write it at the same time that I read and modify Gate's source to introduce the channels (see my previous entry).

While working on Gate's code, I found an interesting thing.
The whole client-server communication is centralized on the "server" side, which implements… The server itself, of course, but also a "proxy" that is able to talk with it.

Great! It means that extracting the communication protocol itself will only involve changes in the server package. No shotgun surgery in the whole project.

The currently implemented protocol is go's RPC client, which works via HTTP. Not a bad choice; but I really would like to propose at least one alternate implementation, using ZMQ.

So the first thing is: move the current implementation out of the server implementation (which should concentrate on just implementing server stuff, not protocol), and out of its proxy implementation.

Well… the role of the proxy is only to send things to the server; but I like the symmetry, similar to ISO's stack: one layer talks to its counterpart at the same level. Hence, the server talks with its proxy, and each one should contain a "channel" object; each channel object should be able to talk to another channel object.

Now committing, pushing, publishing… and going to bed :-)

Reading Gate's code

I needed to read again Gate's code to see how much Pwd diverged from it. Remember: Gate was a Pwd port, written some time ago, but not really maintained; whereas Pwd continued to evolve.

The most striking differences are:

• Pwd supports named pipes and low-level sockets as client-server channels. (There was also a ZMQ attempt, not finished.) On the other hand, Gate only supports RPC on HTTP. I need to extract the "channels" behaviour, maybe with another naming since "channel" is a native Go concept. Then, I may want to implement a ZMQ "channel".
• Pwd started to support a web interface, never finished. Gate could use one, too — but only if I intend to finish it (i.e. not now).

Otherwise, the porting was quite fidel to the original. Maybe too much; the server start and synchronization method uses the same kind of algorithm: start the server process, and try to open a socket until it answers and abort if it takes too long. Not pretty! That's why I want to use ZMQ instead: it should better hide those gory details.

To be continued…

Cad reboot

Holidays are over.

Now is time to start again that blog in earnest.

I plan to post at least once a week. To do so, I need to tighten a bit the topic. This blog is about my pet projects; but I own quite a lot of those. I intend to focus on one project for the moment.

The chosen project is: Gate, a port of Pwd to golang.

A few reasons:

• It is written in Go, and I need to update my skill in that language, because I am interested in knowing more than one OO language and culture.
• Eiffel, and especially Liberty Eiffel, is as good as dead. I still dream of reviving it — although I love the language, I am afraid that won't happen; in the last few years, nobody has taken time to build the needed community (me included, mea maxima culpa).
• I need a more solid design, and I intend to rewrite Gate with that aim in mind
• I need to write blog posts and that is a good opportunity :-)

So, if you are interested in some language learning and some (useful) software building… Stay tuned.

Merry hacking!

Yahoo! pipes is closing up

I rely quite a bit on Yahoo! pipes: that tool is useful to aggregate feeds and spit only the interesting entries.

Therefore their announcement that they were closing up took me by surprise.

I just coded a replacement. Don't expect bells and whistles. It is a trimmed-down version. Should I say bare-bones?

Let me introduce PyCeed. It provides:

• A web server that serves configuration pages and aggregated feeds
• and nothing else :-)

That server is meant to be self-hosted. I intend to do just that, with an actual feed reader in front of it.

Stay tuned!

Happy hacking.

JSON/R, epilogue

The great thing in the Java / Open Source world, is the ability one has to create and share artifacts.

JSON/R is now available, not only on Github (see my previous article), but also as a compiled artifact in Maven Central, hence only at a few XML lines away from your pom.

I hope it may be useful to someone.

JSON/R - the code

JSON/R now has a live repository: https://github.com/cadrian/jsonref

The license is MIT - very liberal. Improvements are welcome.

By popular demand, here is an example of a JSON/R string:

<{"timestamp":"3915-07-10T12:00:00.000","value":"a","class":"net.cadrian.jsonref.Pojo","reference":$1},{"timestamp":null,"value":"b","class":"net.cadrian.jsonref.Pojo","reference":$0}>

JSON/R

JSON is well and good, but it has one shortcoming: the management of object references. When you have to serialize an object graph, with object cycles, JSON is lost.

My client needed to serialize such a graph. So I wrote a JSON extension, dubbed "JSON/R" (JSON with references).

I wanted the parser to stay as pretty as the standard JSON's (see a previous article), with the same property: a parser with no backtrack.

I added two concepts: "heap" and "reference".

A "heap" is just an array with different brackets (< and >); its usage is that the objects in it are "referenced" by their position in the array. By convention, the root object of the graph is the first one, in other words, $0.

A "reference" is an index into the heap: noted $<integer>.

I should write a free parser too. The concept is basic enough to be quite useful.

Introducing yet another Continuous Automation Design

At work, they have server blades, terabytes of disks, multiple backups and failure plans, big architectures with continuous integration servers.

At home, I have a small ARM server (a Cubieboard II) running 24/7; and a family PC that usually runs only during the day.

Despite of the smallness of my home computing resources, I would like to work on my personal projects with a proper continuous automation system.

I have the following requirements:

1. I want the system running at home, no SaaSS. Exit Travis and co.
2. The system must be hosted on the server (ARM), and use as little resources as possible. Exit Jenkins, Teamcity, and others.
3. I also want something a bit more sophisticated than a bunch of shell scripts glued together with a cron table.
4. I want something distributed to make use of all my machines
5. I want to code!

My solution: yaCAD.

• Written in C, object-ish. Using 0mq, sqlite, JSON.
• The server ("core") is as small as possible: scheduling and task distribution.
• The slaves ("runners") will perform the actual task.
• I also want a GUI (CGI?) for administration.

Only the "core" server is already coded. I still have to hack the "runner" slaves, and the GUI.

That's my pet project of the moment. Written in GPLv3, I'd be glad to accept any form of contribution.

JSON

JSON (JavaScript Object Notation) is a language that describes structured data.

It features:

• Simple "atomic" data: numbers, strings, boolean, and a "null" value.
• Arrays, i.e. sequences of successive values.
• Dictionaries ("objects" in JSON parlance), which simply are bunches of values indexed by string keys.

That's all. And that's enough.

I fell in love with JSON a few years ago. The reasons are both functional and technical.

Functionally, JSON is the result of what XML failed to be: a simple data exchange format with a straightforward ecosystem (no DTD, schema, validation, tools over layers and layers of tools and libraries), no frivolities (e.g. no data vs. attribute wars).

Technically, the JSON parser is also very simple. Its grammar is a straightforward descending grammar with no rollbacks. It means that at each point in the JSON text, given the past and current letter, you know exactly what may follow. If it fails: syntax error, out.

Since I love that format I had to build my own parser library: YacJP. That library allows to read and write JSON data using any kind of UTF-8 character streams, including simple C strings, file descriptors, and POSIX files.

It was the first of a few projects that followed in the same "object-like" style.

There will be more to follow... stay tuned!

ExPerience

A few days ago, I came across a very cool tool: Petit.

That tool helps filter standard Linux logs. Since I have my own (small) server, I need that kind of tool.

My server is really small: a Cubieboard, with a Debian distro. It has an internet façade so I must at least cursolily read its logs. That's, hem, tedious. I am not at ease with the admin side of machines.

Back to the topic: Cubieboard is good, but it is not fast. Petit is written in Python; I needed something faster. So I wrote ExP, a Petit clone in plain C.

Well, "plain" is not really accurate. I am an object hacker deep inside. My C code is written in a modular fashion with a lot of encapsulation (static functions) and interfaces (structs with functions).

I find that way of programming usually enough to implement a basic OO design; the remaining of the object paradigm is usually useful in very few cases. When I need that, I switch to Eiffel, but most people won't follow there. Oh, well...


The project | Debian packages