<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
 
 <title>thinking_on thinking</title>
 <link href="https://thinkingonthinking.com/atom.xml" rel="self"/>
 <link href="https://thinkingonthinking.com/"/>
 <updated>2026-11-22T16:47:47+00:00</updated>
 <id>https://thinkingonthinking.com</id>
 <author>
   <name>Patrick Mulder</name>
   <email>mulder.patrick@gmail.com</email>
 </author>

 
 <entry>
   <title>Turning a Man with a Dog into Pixel Art by diffusion</title>
   <link href="https://thinkingonthinking.com/tessera-tiny-pixel-diffuser/"/>
   <updated>2026-11-22T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/tessera-tiny-pixel-diffuser</id>
   <content type="html">&lt;p&gt;Sunday morning, coffee, Hugging Face’s featured models. Today’s &lt;a href=&quot;https://huggingface.co/datasets&quot;&gt;pick&lt;/a&gt; led to a pixel-art generator, and I got stuck on a dumb thought: Can it draw a 16x16 pixel picture of a man with a dog? I tried to read the architecture, and it requires a bit more learning to finally answer. Let us start with part 1.&lt;/p&gt;

&lt;p&gt;The featured model on Hugging Face is &lt;a href=&quot;https://huggingface.co/thejagstudio/tessera-small&quot;&gt;Tessera-small&lt;/a&gt;. It is a 47.6M-parameter model by Jagrat Patel, and its &lt;a href=&quot;https://huggingface.co/cfahlgren1/tessera-small-onnx&quot;&gt;in-browser ONNX port&lt;/a&gt; by &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;cfahlgren1&lt;/code&gt;, which powers the &lt;a href=&quot;https://huggingface.co/spaces/cfahlgren1/slack-moji&quot;&gt;Slack Moji&lt;/a&gt; demo. The canvas is so small that you can count every pixel and watch quantization, sampling and training data each leave their mark.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;/static/images/tessera_mode_impact_summary.png&quot;&gt;&lt;img src=&quot;/static/images/tessera_mode_impact_summary.png&quot; alt=&quot;Tessera rendering quality and sampler mode impact&quot; /&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Same prompts and seeds, five sampler settings. Click for the full-size chart.&lt;/em&gt;&lt;/p&gt;

&lt;h2 id=&quot;a-pixel-called-mask&quot;&gt;A pixel called MASK&lt;/h2&gt;

&lt;p&gt;Tessera is a masked diffusion model over discrete tokens. To see what that means, think of a picture as a collection of pixels, much like a text is a collection of words. In Tessera, each pixel is one of 512 colours, just like each word in a language model comes from a fixed vocabulary.&lt;/p&gt;

&lt;p&gt;Tessera uses one extra token that is not a colour, the mask token. To understand why, take the sentence “The dog ___ a duck.” You can guess the missing word: chases, maybe bites. Tessera learns the same game for pictures: hide some pixels, guess the missing colours. During training, the &lt;a href=&quot;https://sander.ai/2026/08/24/continuous-dlms.html&quot;&gt;forward process of discrete diffusion&lt;/a&gt; gradually turns some colours into &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;[MASK]&lt;/code&gt; until the whole image is in the absorbing, fully masked state.&lt;/p&gt;

&lt;p&gt;To draw a new picture, it starts with every pixel masked and fills the canvas step by step. The model does not learn &lt;em&gt;where&lt;/em&gt; to put the man or the dog. It learns &lt;em&gt;what colour might belong in a pixel&lt;/em&gt; after seeing the prompt and all the colours already known around it.&lt;/p&gt;

&lt;h2 id=&quot;where-the-chessboards-come-in&quot;&gt;Where the chessboards come in&lt;/h2&gt;

&lt;p&gt;To draw a sprite:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;A small tokenizer splits the prompt into 24 text tokens, padded if necessary. A tokenizer and an embedding model are separate pieces: the tokenizer makes chunks of text, while CLIP turns those chunks into number vectors. CLIP is the frozen text encoder here; Tessera does not change its weights. In a high-dimensional embedding space, text with similar meaning tends to end up near one another because the model learned from text and image pairs. CLIP also uses this arrangement for both of the pair’s items, unlike &lt;a href=&quot;https://thinkingonthinking.com/yasiv-book-graph/&quot;&gt;the Yasiv graph&lt;/a&gt; I once wrote about: in that graph, books were nodes and co-purchases formed visible edges. Here, a text chunk is itself a point in a much larger space, and there are no edges to inspect.&lt;/li&gt;
  &lt;li&gt;The 24 text tokens and the 256 pixel tokens enter one bidirectional transformer with an hourglass layout: 2 blocks at full resolution, 8 blocks on 2×2 patch-merged tokens, then 2 more at full resolution. Merging 2×2 patches turns the 16×16 canvas into an 8×8 grid, so 8 of the 12 blocks literally work on a chessboard.&lt;/li&gt;
  &lt;li&gt;Starting from an all-&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;[MASK]&lt;/code&gt; canvas, the model gives every pixel a distribution over colours. It commits the pixels it is most sure about, then gives the remaining positions another prediction. A few large steps are difficult because pixels committed in parallel are only conditionally independent: each head of the duck may look plausible on its own, even if the two heads together do not. That is why careful samplers spread the commitments out and use spatial checks instead of filling the grid in one go.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Training was small too: 82,342 steps of 512 sprites, about 5.5 hours on one NVIDIA RTX PRO 6000 Blackwell. That’s roughly one Sunday morning. On held-out data the model gets 78.8% of masked pixels right.&lt;/p&gt;

&lt;h2 id=&quot;quantization-what-actually-shrinks&quot;&gt;Quantization: what actually shrinks&lt;/h2&gt;

&lt;p&gt;To run Tessera in a browser with no inference server, the ONNX port stores the weights as int8. The download drops from 224 MB (the fp16 export) to 115 MB. Both numbers include the CLIP text encoder: the int8 denoiser is 51 MB and the int8 text encoder is 64 MB. So if you multiply 47.6M parameters by 2 bytes, get about 95 MB and wonder about the gap, the gap is CLIP.&lt;/p&gt;

&lt;p&gt;Some details from the port that I liked:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Only storage is int8. Each weight matrix has one scale per output channel, and the weights are turned back into fp32 at inference time, so the math still runs in fp32.&lt;/li&gt;
  &lt;li&gt;The convolution weights stay fp16, because per-channel int8 gave wrong WebGPU results for 4D conv weights in onnxruntime-web 1.22. That’s only about 0.3M of the 47.5M denoiser parameters.&lt;/li&gt;
  &lt;li&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;Pow(x, 2)&lt;/code&gt; was rewritten as &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;x*x&lt;/code&gt;, because the original could return NaN on some GPUs for negative inputs.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The outputs barely change. On 8 test prompts, the int8 build and the fp16 export pick the same top colour for 91–99% of pixels. The repo reports about 1 second for 4 sprites on a laptop with WebGPU. On my machine a batch took about 3 seconds.&lt;/p&gt;

&lt;p&gt;So the precision of the weights isn’t where image quality is won or lost. Sampling is.&lt;/p&gt;

&lt;h2 id=&quot;sampling-where-the-quality-goes&quot;&gt;Sampling: where the quality goes&lt;/h2&gt;

&lt;p&gt;The chart runs the same prompts and seeds through five settings:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;&lt;strong&gt;Fast / naive:&lt;/strong&gt; 4 steps, no spatial NMS&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;No NMS:&lt;/strong&gt; 16 steps, no spatial NMS&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Browser default&lt;/strong&gt; (what Slack Moji uses): 16 steps, guidance 3, with NMS&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Advanced:&lt;/strong&gt; 24 steps, guidance ramping up to 4.0 on a cosine curve&lt;/li&gt;
  &lt;li&gt;&lt;strong&gt;Advanced + polish:&lt;/strong&gt; the same, plus two extra passes that re-mask and redraw the least confident 8% of pixels&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The browser default is already a careful sampler. It commits the most confident pixels first on a cosine schedule, adds annealed Gumbel noise, and runs 3×3 non-maximum suppression, so neighbouring pixels that are both unsure don’t get committed in the same step. Turning NMS off is what makes the first two columns speckle. With only 4 steps, too many neighbours lock in before they can see each other: the cat breaks into 39 colours and the rubber duck grows a second head.&lt;/p&gt;

&lt;p&gt;At the other end, more steps and the polish passes clean up stray pixels. The health potion goes from 36 colours in naive mode to 23 at the default and 14 after polishing.&lt;/p&gt;

&lt;p&gt;The chart also prints a confidence number under each sprite. It rises from 65–78% (naive) to 80–84% (default) to 91–95% (polished). Take that with a grain of salt: the polish passes redraw exactly the least confident pixels, so the average goes up partly by design. The colour counts and the pictures are the better evidence.&lt;/p&gt;

&lt;h2 id=&quot;training-why-a-man-who-is-walking-with-a-dog-has-no-dog&quot;&gt;Training: why “a man who is walking with a dog” has no dog&lt;/h2&gt;

&lt;p&gt;Even with 24 steps and polish, “a man who is walking with a dog” gives me a walking man and no dog. More sampling doesn’t fix it, because the problem comes from training.&lt;/p&gt;

&lt;p&gt;Tessera was trained for 30 epochs on &lt;a href=&quot;https://huggingface.co/datasets/Angais/pixel-art-atlas-1.4m&quot;&gt;Pixel Art Atlas 1.4M&lt;/a&gt;. Those 1.4 million sprites don’t come from old games. They were generated from text prompts with Krea 2 Turbo and a pixel-art LoRA, and almost every one shows a single centred subject. The dataset card also warns that the captions describe what the generator was asked to draw, and that colours, counts and other details may not match the picture. So the model saw plenty of prompts that mention two things, attached to pictures that often show one.&lt;/p&gt;

&lt;p&gt;It isn’t that the grid is too small. A 4-pixel stick man next to a 3-pixel dog would fit in 16×16. The model simply never learned to draw two subjects. My reading of what happens: once it commits to a centred figure in the first few steps, the rest of the canvas becomes background.&lt;/p&gt;

&lt;p&gt;Two workarounds do get a dog, shown at the bottom of the chart. You can generate “walking man” and “small white dog” separately and put them side by side on a 32×16 canvas. Or you can use inpainting, which Tessera supports, and seed the left and right halves of the canvas with man and dog pixels.&lt;/p&gt;

&lt;h2 id=&quot;what-i-took-away&quot;&gt;What I took away&lt;/h2&gt;

&lt;p&gt;int8 moved the model from a GPU server into a browser tab with almost no change in what it draws. Sampling choices (steps, NMS, re-masking) changed the pictures far more than quantization did. And neither can add something the training data never showed: in a world of single-subject 16×16 sprites, a man walking a dog is two sprites stitched together.&lt;/p&gt;

&lt;p&gt;If you want to play with it yourself: Tessera-small is licensed CC BY-NC 4.0, so it’s free for personal and research use with credit, but commercial use needs a separate licence from the author.&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Voyage en France pendant le deconfinement</title>
   <link href="https://thinkingonthinking.com/voyage-en-france/"/>
   <updated>2020-06-26T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/voyage-en-france</id>
   <content type="html">&lt;p&gt;Dans les airs, il n’y a pas d’avions. Sur la route, il y a moins des voitures. Et sur l’eau, il n’y a que de petits bateaux où, normalement, on voit des bateaux de croisières ou des super yachts.&lt;/p&gt;

&lt;p&gt;Nous sommes le vendredi 19 juin 2020. Les frontières de l’UE sont ouvertes depuis 4 jours. Pendant les derniers mois, il n’était pas facile de comprendre si on pouvait traverser les frontières. Et quand on en passe une, si on doit rester confiné ou pas.&lt;/p&gt;

&lt;p&gt;En général, voyager dans les temps du coronavirus est devenu plus compliqué. On doit prendre assez de de masques dans la voiture et à beaucoup d‘endroits se laver les mains avec un gel de désinfection. Heureusement, conduire de longues distances en voiture est plus agréable grâce aux podcasts.&lt;/p&gt;

&lt;p&gt;Par exemple, une émission traite de la &lt;a href=&quot;https://pages.rts.ch/la-1ere/programmes/cqfd/13-05-2020&quot;&gt;fatigue de cerveau due au télétravail&lt;/a&gt;.  Une autre emission parle d’&lt;a href=&quot;https://www.zeit.de/zustimmung?url=https%3A%2F%2Fwww.zeit.de%2Fgesellschaft%2F2020-01%2Feva-schulz-interviewpodcast-alles-gesagt&quot;&gt;Eva Schulz&lt;/a&gt;, une jeune journaliste qui discute des problemes due à la surconsommation pasive de la télé . Elle mentionne l’oeuvre de Pierre Bourdieu qui analyse les mécanismes par/selon/à travers lesquels la TV produit de moins en moins de pluralité. Dans un dernier podcast, pendant ces 900km, (il s‘agit de) il s‘agissait de la vie du  pilote &lt;a href=&quot;https://www.europe1.fr/emissions/Au-coeur-de-l-histoire/le-saviez-vous-jean-mermoz-a-inspire-a-saint-exupery-son-livre-le-petit-prince-3971003&quot;&gt;Jean Memoz&lt;/a&gt;, qui a vécu dans la première partie du 20ème siècle. En écoutant cette histoire, on apprend les difficultés que les pionniers de l’aviation ont eues/éprouvées/connues.&lt;/p&gt;

&lt;p&gt;Avec ces informations on peut oublier pour quelques heures les défis du coronavirus. Mais les premières impressions de la vie en France donne une nouvelle impression de la réalité actuelle. Dans la ville de Cannes, on voit de magasins fermés. Il n’y a pas de touristes dans la rue, pas de queues devant les musées. Il y a des affiches qui recommandent de consommer régional (et j‘en suis d‘accord). Comprendres les vendeurs derrière un masque, ce n’est pas facile, mais comme l‘affirme une autre affiche, porter le masque représente un acte de solidarité. Je ne sais pas si les masques peuvent arrêter la propagation du virus, mais si on peut relancer la vie économique grâce à eux, c’est important aussi.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Le vent nous portera</title>
   <link href="https://thinkingonthinking.com/need-writing-again/"/>
   <updated>2019-01-10T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/need-writing-again</id>
   <content type="html">&lt;p&gt;While driving during the X-mas holidays, I listened to a radio interview about a poetry in music tracks. Someone read the lyrics of “Le vent nous portera” (The wind will transport us) by a French band “Noir desir”.
The internet makes sharing so easy, even small thoughts like this one. In a sense, the internet is like the wind that transports thoughts.
So, here the lyrics of the &lt;a href=&quot;https://www.youtube.com/watch?v=NrgcRvBJYBE&quot;&gt;song&lt;/a&gt; by Noir desir:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;Je n&apos;ai pas peur de la route
Faudrait voir, faut qu&apos;on y goûte
Des méandres au creux des reins
Et tout ira bien
Le vent l&apos;emportera
Ton message à la grande ourse
Et la trajectoire de la course
A l&apos;instantané de velours
Même s&apos;il ne sert à rien
Le vent l&apos;emportera
Tout disparaîtra
Le vent nous portera
La caresse et la mitraille
Cette plaie qui nous tiraille
Le palais des autres jours
D&apos;hier et demain
Le vent les portera
Génétique en bandoulière
Des chromosomes dans l&apos;atmosphère
Des taxis pour les galaxies
Et mon tapis volant dis?
Ce parfum de nos années mortes
Ceux qui peuvent frapper à ta porte
Infinité de destin
On en pose un, qu&apos;est-ce qu&apos;on en retient?
Le vent l&apos;emportera
Pendant que la marée monte
Et que chacun refait ses comptes
J&apos;emmène au creux de mon ombre
Des poussières de toi
Songwriters: Bertrand Lucien Cantat / Denis Guy Marc Barthe / Gay Serge Marius Pierre Teyssot / Jean-Paul Roy
Le vent nous portera lyrics © Universal Music Publishing Group
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;
</content>
 </entry>
 
 <entry>
   <title>Eclipses and programming</title>
   <link href="https://thinkingonthinking.com/eclipses-and-programming/"/>
   <updated>2017-09-07T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/eclipses-and-programming</id>
   <content type="html">&lt;p&gt;When it comes to embedded systems, working with compilers gets important sooner or later.&lt;/p&gt;

&lt;p&gt;So, it was interesting to listen to Robert C. Martin about differences between C, Objective C and C++ (and some general history of programming ).&lt;/p&gt;

&lt;iframe width=&quot;560&quot; height=&quot;315&quot; src=&quot;https://www.youtube.com/embed/ecIWPzGEbFc&quot; frameborder=&quot;0&quot; allowfullscreen=&quot;&quot;&gt;&lt;/iframe&gt;

&lt;p&gt;To program with C and C++, it usually takes a good IDE. While easy to use, the Arduino IDE is too simple for many professional programmers. One possible next step for embedded development is &lt;a href=&quot;https://www.eclipse.org/&quot;&gt;Eclipse&lt;/a&gt;. As an additional motivation to explore Eclipse closer, the Eclipse open-source ecosystem. For example, the slide deck by &lt;a href=&quot;https://qconlondon.com/london-2008/qconlondon.com/dl/qcon-london-2008/slides/ErichGamma_qcon2008.pdf&quot;&gt;Erich Gamma&lt;/a&gt; on the history of Eclipse is quite interesting. There is also &lt;a href=&quot;https://www.eclipsecon.org/europe2017/&quot;&gt;EclipseCon&lt;/a&gt; which shows how the IDE evolves into different directions.&lt;/p&gt;

&lt;p&gt;Exploring plugins can be a good place to start:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Eclipse &lt;a href=&quot;https://eclipse.org/cdt/&quot;&gt;CDT&lt;/a&gt; and &lt;a href=&quot;https://eclipse.org/jdt/&quot;&gt;JDT&lt;/a&gt; are probably the main drivers of Eclipse. These plugin systems focus on C/C++ and Java development respectively.&lt;/li&gt;
  &lt;li&gt;Eclipse &lt;a href=&quot;http://www.pydev.org/&quot;&gt;PyDev&lt;/a&gt; is an interesting extension for working with Python projects.&lt;/li&gt;
  &lt;li&gt;While the CDT has an Arduino extension as well, the &lt;a href=&quot;http://eclipse.baeyens.it/&quot;&gt;Arduino plugin&lt;/a&gt; has all kinds of interesting additions.&lt;/li&gt;
  &lt;li&gt;The Eclipse &lt;a href=&quot;https://github.com/boothen/Json-Eclipse-Plugin&quot;&gt;JSON plugin&lt;/a&gt; is interesting to work with data projects.&lt;/li&gt;
  &lt;li&gt;The Eclipse &lt;a href=&quot;https://github.com/ceylon/ceylon-ide-eclipse&quot;&gt;Cylon plugin&lt;/a&gt; and Eclipse &lt;a href=&quot;https://github.com/RustDT/RustDT&quot;&gt;Rust plugin&lt;/a&gt; seem interesting to explore.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Now, according to some discussions in the internets, Eclipse is not the latest state-of-the-art IDE. Agreed. For web and other kinds of software projects, Eclipse may feel slow. And features are not yet finished or simply unclear. But as technology and data projects evolve, Eclipse might still be relevant for quite some years to come.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Finding Beethoven on Coursera</title>
   <link href="https://thinkingonthinking.com/beethoven-on-coursera/"/>
   <updated>2017-08-21T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/beethoven-on-coursera</id>
   <content type="html">&lt;p&gt;Summer time is great. Time for reading, time for discoveries, time to reflect. It is also good to look at the bigger picture of physics, software or music. Here are some books that I touched:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;https://www.goodreads.com/book/show/25602451-losing-the-signal&quot;&gt;“Losing the Signal: The Untold Story Behind the Extraordinary Rise and Spectacular Fall of BlackBerry”&lt;/a&gt;: This book is about the history of BlackBerry. One of the company’s founder &lt;a href=&quot;https://en.wikipedia.org/wiki/Mike_Lazaridis&quot;&gt;Mike Lazaridis&lt;/a&gt; had access to an early version of &lt;a href=&quot;https://en.wikipedia.org/wiki/Mobitex&quot;&gt;Mobitex&lt;/a&gt; - the predecessors of wireless data networks. The book is a very good overview about the beginnings and developments of mobile data networks in the 90ies (For example paging, Motorola, Palm, US Robotics, … ).  The history of BlackBerry also shows the impact of data networks on the evolution of mobile devices.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;https://www.goodreads.com/book/show/29208951-joseph-fourier?from_search=true&quot;&gt;“Joseph Fourier by Joseph Arago”&lt;/a&gt;:  For electronic communication, we look at signals in two different ways: In “time-domain” or “frequency-domain”. The mathematical foundations for these perspectives were discovered by Joseph Fourier. With this book, I hoped to learn more on the origins of the mathematics of signal processing. Interestingly, not much of mathematics were discussed in the book by &lt;a href=&quot;https://en.wikipedia.org/wiki/Fran%C3%A7ois_Arago&quot;&gt;Joseph Arago&lt;/a&gt;. Instead, it was surprising that Fourier traveled to Egypt  and reflected about the mechanisms of heat for some years. Prbobably triggered by the look of the desert, he started discovering the “greenhouse effect” in 1824 (e.g. see &lt;a href=&quot;https://blogs.scientificamerican.com/plugged-in/why-we-know-about-the-greenhouse-gas-effect/&quot;&gt;Scientific Americam&lt;/a&gt; ).&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;https://www.goodreads.com/book/show/31823677-tools-of-titans&quot;&gt;“Tools of the Titans”&lt;/a&gt;: I discovered this book by Tim Ferris because it is  on the NYTimes bestseller’s list 2017. Also, I’ve heard about Tim Ferris before. Ferris’ based this book on Ben Franklin’s quote: “Early to bed and early to rise makes a man healthy, wealthy, and wise.” So, the book indeed makes you aware that extreme characters wake up earlier, focus better and tolerate more their weaknesses (and use them wisely). The book is also interesting with respect to creative writing and probably my motivation for this blog post.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Well, I started some more books and other things. But right now, I am most fascinated by some courses on Coursera, esp. the &lt;a href=&quot;https://www.coursera.org/learn/beethoven-piano-sonatas/lecture/E2dNG/re-shaping-the-sonata-op-26&quot;&gt;Beethoven Sonata Lecture by Jonathan Bliss&lt;/a&gt;. there are so many interesting puzzles and ideas in classical music. And audio is full of mathematics too, as well as circuits and hardware, as well as software (and a bit of Fourier analysis).&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>I broke the 3d printer</title>
   <link href="https://thinkingonthinking.com/I-broke-the-3d-printer/"/>
   <updated>2017-03-25T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/I-broke-the-3d-printer</id>
   <content type="html">&lt;p&gt;Two weeks ago at the &lt;a href=&quot;http://www.modellbauclub-mcsm.de/Munich-Maker-Space&quot;&gt;MCSM Modelbauclub&lt;/a&gt;, I broke the 3D printer. Well, I damaged the printing plate by messing up the calibration of the printing head. Luckily, last week we could repair the damage, but it took some time.&lt;/p&gt;

&lt;p&gt;It is no surprise hat with hardware things can break, with software you can just restart the computer. This makes the feeling of developing hardware a bit different from the software development experience. For hardware, gaining a feeling for a construction beforehand is important. There is no such concept as test-driven development.&lt;/p&gt;

&lt;p&gt;The outcome of a construction depends on the preparation. Instead of starting the 3D print process without a feeling for the position of the 3D printing head, it would have been important to listen to another’s opinion. Or, to have some kind of picture of the position of pieces in space.&lt;/p&gt;

&lt;p&gt;For large constructions, this can be done with simulation. Even for electronics and circuits simulation can avoid a lot of trial-and-error. Planning and preparation is also something German engineers are famous for. At MCSM, there are a number of retired engineers with many years of engineering experience. It is interesting to see how they have a feeling for a construction and patience to prepare a construction.&lt;/p&gt;

&lt;p&gt;I guess experiences of failure like this with 3D printing are important to deal with for engineers and technologists. We often blame someone else or some circumstances when things break or don’t work. In these cases, it is always nice to hear people such as &lt;a href=&quot;https://www.youtube.com/watch?v=Vhk_5UBZDQA&quot;&gt;Jack Ma founder of Alibaba&lt;/a&gt; explain how failure can help you grow as human.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/3dprinter.jpg&quot; /&gt;&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Learning from a Graph</title>
   <link href="https://thinkingonthinking.com/yasiv-book-graph/"/>
   <updated>2017-01-23T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/yasiv-book-graph</id>
   <content type="html">&lt;p&gt;&lt;a href=&quot;http://www.yasiv.com/&quot;&gt;Yasiv&lt;/a&gt; is a wonderful tool to browse product data at Amazon. By looking at a graph of sales data, where does our book &lt;a href=&quot;http://embeddednodejs.com&quot;&gt;Node.js for embedded systems&lt;/a&gt; stand?&lt;/p&gt;

&lt;p&gt;The graph of book sales Amazon can provide some insights:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/amz_nodejs_embedded.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The graph shows different kinds of products from the Amazon product catalog. The lines between nodes shows how books are combined in a shopping cart.&lt;/p&gt;

&lt;p&gt;In the middle, the yellow circle shows the book “Node.js for Embedded Systems”. It connects with a number of different product categories: Raspberry Pi, Arduino, BeagleBone, Robots, Design and Web Development.&lt;/p&gt;

&lt;p&gt;Let’s look into details. First, people buy this book together with books on materials on Raspberry Pi and Arduino (cyan circles top left). Single board computers are indeed an interesting start with embedded systems. Books on Beaglebone are also connected to this (see orange circle).&lt;/p&gt;

&lt;p&gt;Second, there is a whole book category on Robots (red circle). From the data, it looks like robot books are more connected to Raspberry Pi and Arduino, than they are to Beaglebone. It seems our book could add an interesting link here, but it is not yet what people see. Somehow in this direction of machines, books are written on computer architecture and FPGA’s as well.&lt;/p&gt;

&lt;p&gt;Third, there is a whole category of books on design of user experiences for connected products. Interestingly, only few books link design with hardware for IoT. Two books are “Wearable Sensors: Fundamentals, Implementation and Applications” (2014) and “Learning Internet of Things” (2015).&lt;/p&gt;

&lt;p&gt;Last, there is a large category of books on writing web applications with Node.js or JavaScript in general (blue circles bottom). This is the good news for “Node.js for Embedded Systems”: It seems to be able to link hardware with web applications.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Modern arts tour in Houston</title>
   <link href="https://thinkingonthinking.com/modern-arts-in-houston/"/>
   <updated>2016-10-16T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/modern-arts-in-houston</id>
   <content type="html">&lt;p&gt;Recently I had the luck to visit Houston. Maybe you are curious to hear about arts in Houston, as I was?&lt;/p&gt;

&lt;p&gt;My arts tour started in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Houston_Museum_District&quot;&gt;Houston museum district&lt;/a&gt;. There is “CAMH”, the &lt;a href=&quot;http://camh.org/&quot;&gt;Museum for Contemporary Arts Houston&lt;/a&gt; where some Houston based artists were on show. First, I saw paintings and sculptures by &lt;a href=&quot;http://www.independenceartstudios.com/thedra/artists-statement&quot;&gt;Thedra Cullar-Ledford&lt;/a&gt;. Thedra Cullar-Ledford was diagnosed with cancer in 2013. Her pieces tell a story of battles with therapies and tumors, but also about genders, identity and sexuality. As Houston is a center for cancer research, her pieces could communicate more than what goes on behind research and hospitals.&lt;/p&gt;

&lt;p&gt;A bit different stories were told by &lt;a href=&quot;Susie Rosmarin&quot;&gt;Susie Rosmarin&lt;/a&gt;. Form should follow function in good design.. In Rosmarin’s pieces, the function is taken out again from signals and pixels. All we are left with are &lt;a href=&quot;http://www.houstonpress.com/arts/100-creatives-susie-rosmarin-6397858&quot;&gt;patterns and colors&lt;/a&gt;. While her work is analog, I found it interesting to reflect on her ideas for digital worlds as well.&lt;/p&gt;

&lt;p&gt;From the CAMH, my tour continued to the &lt;a href=&quot;https://www.menil.org/&quot;&gt;Menil Collection&lt;/a&gt;. The impressive building was designed by &lt;a href=&quot;http://www.rpbw.com/project/25/the-menil-collection/&quot;&gt;Renzo Piano&lt;/a&gt;. Unfortunately, Mondays’ and Tuesdays’ the main collection is closed. All I could visit was the &lt;a href=&quot;http://www.rothkochapel.org/&quot;&gt;Rothko Chapel&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/rothko_chapel.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The Rothko Chapel is very special though. It is one of the few places I know where all religions can meet. It feels warm and calm to connect with other cultures through aesthetics.&lt;/p&gt;

&lt;p&gt;Leaving the Rothko Chapel, I moved on to the &lt;a href=&quot;http://www.gremillion.com/thomaszitzwitz/&quot;&gt;Gremillion Gallery&lt;/a&gt; to see works of an artist friend from Germany: &lt;a href=&quot;http://www.zitzwitz.com/&quot;&gt;Thomas Zitzwitz&lt;/a&gt;. As a trained computer scientist and artist, Thomas’s work provides interesting views on abstract art and the nature of light. At Gremillion’s I also got an interesting overview about appreciation of modern art in Texas. While Houston can be seen as the center, it should be well worth to visit the &lt;a href=&quot;http://www.themodern.org/&quot;&gt;Fort Worth Modern Art museum&lt;/a&gt;. The building by Tadao Ando has been titled as the &lt;a href=&quot;http://www.themodern.org/about-modern/building&quot;&gt;“World’s most beautiful art museum”&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Well, talking about buildings and arts installations, my tour ended at &lt;a href=&quot;http://skyspace.rice.edu/&quot;&gt;Skyspace&lt;/a&gt; by James Turrell.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/turrell_sky.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;As the name suggest, a &lt;a href=&quot;https://en.wikipedia.org/wiki/Skyspace&quot;&gt;sky space&lt;/a&gt; invites you to look at the sky and enjoy the light. This is another example how arts can give you inner peace and happiness.&lt;/p&gt;

&lt;p&gt;The word Texas, which means “friends” in the Caddo language (from “Tejas”) certainly made a good impression on me. Thank you for being so open and friendly!&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>From JavaScript to C</title>
   <link href="https://thinkingonthinking.com/learning-c-for-javascripters/"/>
   <updated>2016-09-30T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/learning-c-for-javascripters</id>
   <content type="html">&lt;p&gt;JavaScript?! Most embedded developers will tell you to learn C and assembler rather sooner than later. Indeed, &lt;a href=&quot;http://embedded.fm/blog/2016/2/25/do-i-need-to-know-assembly-language-to-write-c-well&quot;&gt;Understanding assembler&lt;/a&gt;  and &lt;a href=&quot;http://embedded.fm/episodes/159&quot;&gt;bit-and-byte level programming&lt;/a&gt; are important when you program low-power devices.&lt;/p&gt;

&lt;p&gt;For the background: In &lt;a href=&quot;http://embeddednodejs.com&quot;&gt;embeddednodejs.com&lt;/a&gt;, &lt;a href=&quot;https://twitter.com/SelkeyMoonbeam?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor&quot;&gt;Kelsey&lt;/a&gt; and me argue where and how a high-level scripting langugage (esp. JavaScript) has benefits for developing applications for IoT and physical computing.&lt;/p&gt;

&lt;p&gt;But for sure JavaScript does not solve all problems you might face. When you know JavaScript (or other scripting languages), why and how can you start to (re-)learn C?  Besides that learning  C is a fun inveestment, here are some ideas to help you get started.&lt;/p&gt;

&lt;h1 id=&quot;1-start-with-makefiles&quot;&gt;1) Start with Makefiles&lt;/h1&gt;

&lt;p&gt;Makefiles are essential in any C project. Makefiles wrap calls to a C toolchain such as compiler, linkers and standard libraries. Makefiles help you in learning C code as well. Makefiles can quickly grow complex (if you look at any open source project), but it is important to start simple. For example as simple as:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;cc=gcc
src=main.c

all:
	$(cc) $(src) -o hello
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Without a source file, running “make” results into an error that no source is found. This is a good moment to reflect what you need to learn about C source files and projects&lt;/p&gt;

&lt;h1 id=&quot;2-re-write-code-you-know-in-c&quot;&gt;2) Re-write code you know in C&lt;/h1&gt;

&lt;p&gt;A good advice is to start translating code you know in C. For example, translate this JavaScript call:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;  console.log(&quot;hello, world&quot;);
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;A good C book, such as the &lt;a href=&quot;https://www.goodreads.com/book/show/515601.The_C_Programming_Language&quot;&gt;K&amp;amp;R bible&lt;/a&gt;, will quickly tell you how to write this in C:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;

int main(int argc, char ** argv) {
  printf(&quot;hello, world\n&quot;);
  return 0;
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Running Make gives some ouput. On my Mac, the size of the resulting file is around 8 kB. Compiling the same file for an ARM machine, results into 1 KB of binary file. Ok, running the example is nice, but for learning purposes, it is good to reflect what is inside the binary file.&lt;/p&gt;

&lt;h1 id=&quot;3-disassemble-the-binary-file&quot;&gt;3) Disassemble the binary file&lt;/h1&gt;

&lt;p&gt;Working with C and low-level stuff requires you to understand registers, instructions sets, memory and how machines think. This takes a while, but a good start are disassemblers. Especially, you can compare how compilers translate this simple function above into instructions of a target machine. On my Mac, the &lt;a href=&quot;http://stackoverflow.com/questions/1289881/using-gcc-to-produce-readable-assembly&quot;&gt;command to dissassemble the binary file&lt;/a&gt; is:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; gcc -O2 -S main.c
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This gives:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;  	.section	__TEXT,__text,regular,pure_instructions
  	.globl	_main
  	.align	4, 0x90
  _main:                                  ## @main
  	.cfi_startproc
  ## BB#0:
  	pushq	%rbp
  Ltmp2:
  	.cfi_def_cfa_offset 16
  Ltmp3:
  	.cfi_offset %rbp, -16
  	movq	%rsp, %rbp
  Ltmp4:
  	.cfi_def_cfa_register %rbp
  	leaq	L_str(%rip), %rdi
  	callq	_puts
  	xorl	%eax, %eax
  	popq	%rbp
  	retq
  	.cfi_endproc
  
  	.section	__TEXT,__cstring,cstring_literals
  L_str:                                  ## @str
  	.asciz	&quot;hello, world&quot;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;What these instruction mean is another topic for a blog post all together, but for now, observe the different “sections” in the binary file. These sections are loaded into memory when you run the program. Also, one section keeps the string literal “hello, world”.&lt;/p&gt;

&lt;h1 id=&quot;4-study-code-from-others&quot;&gt;4) Study code from others&lt;/h1&gt;

&lt;p&gt;As with JavaScript, there are plenty of C libraries out there that you can study and learn from. Especially, &lt;a href=&quot;https://www.goodreads.com/book/show/258607.UNIX_Network_Programming&quot;&gt;network programming is done with C&lt;/a&gt; and you could learn by writing a small websockets client or server in C. This will teach you about assembling bytes to communication protocols and working with structs and such. Also, getting a feeling for the C standard libraries is important. OFten, these relate to the target machine that you use (for example study the difference of standard libs for Arduino or your Unix distribution of choice). Another way to learn is about C is builing parsers with Yacc and Lex. This will hopefully be another blog post as well.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Failed software projects</title>
   <link href="https://thinkingonthinking.com/failed-software-projects/"/>
   <updated>2016-08-14T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/failed-software-projects</id>
   <content type="html">&lt;p&gt;While doing some paperwork, I just came across an old invoice for a piece of custom software development in 2011. At that time, someone wanted me to help building a small ecommerce shop. Looking back at the invoice that we paid to a young team in Poland, I am surprised about the amount we paid. Also, I am a bit sad, that the investment in software did not yield any visible returns today.&lt;/p&gt;

&lt;p&gt;In the meanwhile, I have seen this pattern more often: People invest in software development without getting the return they hoped for. Highly motivated, young developers promise to build some solution at a discount price assuming they can work with a new software technology. This would provide them a context to learn something new, while getting paid for it.  For the people who invest money,  the bet is that new frameworks and technologies can help to solve a business need faster. The developers get paid, the result is not as expected and this cycle continues. Some money can get lost on good will, before one of the sides pulls the plug.&lt;/p&gt;

&lt;p&gt;Outcomes of software projects are really hard to judge. In the book &lt;a href=&quot;http://www.goodreads.com/book/show/22104615-the-innovator-s-hypothesis&quot;&gt;“The Innovator’s Hypothesis: How Cheap Experiments Are Worth More Than Good Ideas”&lt;/a&gt;, Michael Schrage quotes Warren Buffet that expected returns of innovation should be related to costs of innovations early on.&lt;/p&gt;

&lt;p&gt;This thought is interesting. Looking back, the software project from 2011 above was an experiment. The developers were able to experiment with some new technologies. The business side was able to learn more on building a software prototype. Looking back, the same results (basically a failed working prototype) could have been achieved wihtout spending money. The software developers might have better experimented with an open-source library or would have written blog posts. The business side could have started with a cheap mailing list or blog before going into development of a web application.&lt;/p&gt;

&lt;p&gt;Now, my situation is a bit different. When working with hardware, it is a bit harder to experiment with the latest open-source technologies (although I try a lot like discussed in &lt;a href=&quot;http://embeddednodejs.com&quot;&gt;embeddednodejs.com&lt;/a&gt;).&lt;/p&gt;

&lt;p&gt;Hardware engineers understand the merits of “never change a running system”. When developing software in a hardware environment, you get an impression that everything moves slower, but on the other hand this can lead to more stable results. But as the Internet changes business models of hardware companies, I am sure we’ll see the question return on how to invest in a software projects.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Design of break out boards</title>
   <link href="https://thinkingonthinking.com/design-ofmodular-hardware/"/>
   <updated>2016-06-13T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/design-ofmodular-hardware</id>
   <content type="html">&lt;p&gt;Breakout boards help to “break out” pins of chips and circuits. In a way, breakout boards are catalysts for hardware development. Learning to design breakout boards is a good start to design more complex circuits and hardware.&lt;/p&gt;

&lt;p&gt;From a software perspective, a break out board feels like a unit test for hardware. Their function is limited to a couple of pins and a couple of square centimeters. Companies such as &lt;a href=&quot;https://www.sparkfun.com/search/results?term=breakout+board&quot;&gt;Sparkfun&lt;/a&gt; or &lt;a href=&quot;https://www.adafruit.com/product/2471?q=breakout%20board&amp;amp;&quot;&gt;Adafruit&lt;/a&gt; offer an impressive number of breakout boards. This shows that, at least for hobby purposes, breakout boards solve a real need.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/bb_boards_sparkfun.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Not only does Sparkfun offer to buy their boards. They also provide a schematic and board layout to actually experiment, copy and modify the board for your purposes. This is quite revolutionary. It is similar to how Linux allows developers to improve toolchains or to write new tools based on the Linux kernel. Similar to C files related Linux, you can share CAD files for breakout boards online.&lt;/p&gt;

&lt;p&gt;Nathan Seidle, founder of Sparkfun, provides a nice overview about the way that the internet revolutionizes hardware development in this talk:&lt;/p&gt;

&lt;iframe width=&quot;560&quot; height=&quot;315&quot; src=&quot;https://www.youtube.com/embed/xGhj_lLNtd0&quot; frameborder=&quot;0&quot; allowfullscreen=&quot;&quot;&gt;&lt;/iframe&gt;

&lt;p&gt;His main message is that our job as engineers is no longer the task to write patents. Our job is making products that sell. At Sparkfun, selling is done via the Internet and this means, that people also buy “information” and “content”. They want to see and understand schematics and board layouts.&lt;/p&gt;

&lt;p&gt;You might think that hardware for IoT is too complex for sharing online. The Tessel 2 is especially an interesting example that it is possible. Not only has the board been designed with open-source software toolchains. It also features a module system that allows software developers a plug-and-play experience for software development. With a simple &lt;a href=&quot;https://github.com/embeddednodejs/tessellib&quot;&gt;Tessel device for Eagle&lt;/a&gt; you can easily start to design your own Tessel modules. And with companies such as &lt;a href=&quot;https://oshpark.com/&quot;&gt;oshpark&lt;/a&gt; you can build your own prototype boards for under 10USD.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/t2_breakout.png&quot; /&gt;&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Keeping ideas simple</title>
   <link href="https://thinkingonthinking.com/book-progress/"/>
   <updated>2016-05-07T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/book-progress</id>
   <content type="html">&lt;p&gt;Writing a book means communicating ideas. While writing, one difficulty I find is dealing with too many ideas at once. Keeping ideas simple and clear is very hard. By chance, I saw a TED talk by Chris Anderson discussing the importance of communicating ideas.&lt;/p&gt;

&lt;iframe src=&quot;https://embed-ssl.ted.com/talks/chris_anderson_teds_secret_to_great_public_speaking.html&quot; width=&quot;640&quot; height=&quot;360&quot; frameborder=&quot;0&quot; scrolling=&quot;no&quot; webkitallowfullscreen=&quot;&quot; mozallowfullscreen=&quot;&quot; allowfullscreen=&quot;&quot;&gt;&lt;/iframe&gt;

&lt;p&gt;&lt;br /&gt;
Chris Anderson mentions 4 strategies to communicate ideas. How could it apply to writing &lt;a href=&quot;http://embeddednodejs.com&quot;&gt;embeddednodejs.com&lt;/a&gt; ?
&lt;br /&gt;&lt;/p&gt;

&lt;h2 id=&quot;strategy-1-focus-on-just-one-major-idea&quot;&gt;Strategy 1: Focus on just one major idea&lt;/h2&gt;

&lt;p&gt;Writing starts because of passion for an idea. Hardware and web technologies (such as JavaScript) motivate me a lot to write &lt;a href=&quot;http://embeddednodejs.com&quot;&gt;embeddednodejs.com&lt;/a&gt;. At the same time, in my daily work at Intel in Germany, I get plenty of interesting challenges that do not directly relate to the book. Solving problems as engineer is great, but it also takes away time from writing. This can result in chapters with too many ideas at once or in chapters with not enough depth. To improve working on single, major ideas, I am very grateful that &lt;a href=&quot;https://twitter.com/SelkeyMoonbeam&quot;&gt;Kelsey Breseman&lt;/a&gt; joins the book as a coauthor. With her experience from the &lt;a href=&quot;https://tessel.io&quot;&gt;Tessel&lt;/a&gt; project, she helps me understand the problems with existing hardware platforms for connected devices.&lt;/p&gt;

&lt;h2 id=&quot;strategy-2-give-a-reason-to-care&quot;&gt;Strategy 2: Give a reason to care&lt;/h2&gt;

&lt;p&gt;With open technologies, there are faster ways to innovate and to collaborate. This changes the way we &lt;a href=&quot;http://thinkingonthinking.com/on-innovation-communities/&quot;&gt;learn, play and work&lt;/a&gt;. It is driven by &lt;a href=&quot;http://thinkingonthinking.com/hardware-cost-reductions/&quot;&gt;new economies of scale&lt;/a&gt; and new ways to particpate in hardware projects on Kickstarter or Indiegogo. The early release of the book has had already a bit of questions, mainly related to selection of boards and hardware. Some readers asked about guidelines for projects. These projects show that we start to see computers as new medium for physical interactions.&lt;/p&gt;

&lt;h2 id=&quot;strategy-3-start-with-the-language-of-the-audience&quot;&gt;Strategy 3: Start with the language of the audience&lt;/h2&gt;

&lt;p&gt;The brain is an important metaphor for the computer. But the brain is just one part of the nervous system of an animal. When you start building systems of connected devices, you get a feeling of the complexities of a nervous system in general. You need devices that are able to plan and to communicate, as well as devices that just act on simple inputs and outputs. So, to explain building computers, it may help to look at metaphors with animals and insects. Also, looking at the language of Arduino might help. Arduino comes with nice concepts in “sketches” to show how hardware works.&lt;/p&gt;

&lt;h2 id=&quot;strategy-4-make-your-idea-worth-sharing&quot;&gt;Strategy 4: Make your idea worth sharing&lt;/h2&gt;

&lt;p&gt;Projects as Tessel and Arduino life from people sharing their experiences. For the book, I try to build the &lt;a href=&quot;http://embeddednodejs.com/chapters.html&quot;&gt;book website&lt;/a&gt; and &lt;a href=&quot;http://github.com/embeddednodejs&quot;&gt;the book examples&lt;/a&gt; around concepts that can be linked and commented on Github. It is hard because there are so many hardware platforms available. And, projects often have special requirements that do not easily map to other projects. Yet, it is the core of open source and with open board schematics and hardware libraries sharing improves day by day.&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Learning about Biohacking</title>
   <link href="https://thinkingonthinking.com/learning-about-biohacking/"/>
   <updated>2016-03-21T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/learning-about-biohacking</id>
   <content type="html">&lt;p&gt;Thanks to &lt;a href=&quot;http://www.dusseiller.ch/labs/?page_id=2&quot;&gt;Marc Dusseiller&lt;/a&gt;, I learnt about interesting intersections between open-source hardware, software and biology at &lt;a href=&quot;http://www.meetup.com/Make-Munich/events/229456071/?_af=event&amp;amp;_af_eid=229456071&amp;amp;https=off&quot;&gt;a meetup last Friday&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;The meetup was about different topics, such as building your own biology lab equipment, open-source disease preventions, up to gardening, hunting and cooking. To understand why these topics are important and interesting, let’s examine the roots of “biohacking”.&lt;/p&gt;

&lt;p&gt;Surprisingly, the word “biology” is not very old. It was only introduced in the early 19th century. The “study of life” is much older though. Back in 4th century BC, &lt;a href=&quot;http://plato.stanford.edu/entries/aristotle-biology/&quot;&gt;Aristotle&lt;/a&gt; started listing attributes of animals. While that sounds simple, these studies resulted in interesting discourses on abstractions and categories.&lt;/p&gt;

&lt;p&gt;Today, students of biology study chemistry, physics, o bit of algorithms and genetics (I would think).  Normally, this requires expensive equipment (I would think). There are different approaches though, as Marc explained.&lt;/p&gt;

&lt;p&gt;With internet and open-source technologies, students can build simple devices themselves to study biology. Googling these ideas resulted into a quote by Mitchel Resnick: &lt;a href=&quot;https://llk.media.mit.edu/papers/mres-wef.pdf&quot;&gt;People don’t get ideas; they make them&lt;/a&gt;. In this context, Marc mentioned an important paper: &lt;a href=&quot;http://web.media.mit.edu/~mres/papers/bbb.pdf&quot;&gt;“Beyond Black Boxes: Bringing Transparency and Aesthetics Back to Scientific Investigation”&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The observations of Resnick apply to science labs outside of biology:&lt;/p&gt;

&lt;blockquote lang=&quot;en&quot;&gt;&lt;p&gt;Over time, the scientific laboratory may, sadly, have become a less beautiful setting in which to work, .... The modern-day student of science is less likely to experience a sense of comfort and delight in their surroundings; and, as Csikszentmihalyi (1996) has observed, the creative enterprise (whether scientific or artistic) is often keenly influenced by just such environmental factors.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;And the paper continues:&lt;/p&gt;

&lt;blockquote lang=&quot;en&quot;&gt;&lt;p&gt;Both the power and the problem with modern scientific instrumentation are reflected in the term “black box” that is commonly used to describe the equipment. Today’s blackbox instruments are highly effective ... But, at the same time, these black boxes ... are bland in appearance (making it difficult for users to feel a sense of personal connection with scientific activity). &lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;To me, it is an advantage of open-source hardware that is much more transparent. The playful aspects of Arduino remind more on working with Lego than with hard to understand electronics at times. It was nice to see the roots of these theories based on Piaget and Papert, resulting into the concept of &lt;a href=&quot;http://web.media.mit.edu/~mres/papers.html&quot;&gt;“lifelong kindergarden”&lt;/a&gt;. The instruments that Marc Dussellier builds will certainly help. Maybe not for people with PhD’s, but hopefully in areas of the world, where there are less means to buy expensive equipment.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/m_dussellier.png&quot; /&gt;&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Building Digital Hardware</title>
   <link href="https://thinkingonthinking.com/building-digital-hardware/"/>
   <updated>2016-02-29T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/building-digital-hardware</id>
   <content type="html">&lt;p&gt;In 2014, there was the first &lt;a href=&quot;http://conferences.oreilly.com/solid/solid2014&quot;&gt;OReilly Solid Conference&lt;/a&gt; about the impact of the “digital” world on the “analog” world. To learn about the foundations for digital fabrication, have a look at this talk by prof. Neil Gershenfeld from the MIT Center for Bits and Atoms.&lt;/p&gt;

&lt;iframe width=&quot;560&quot; height=&quot;315&quot; src=&quot;https://www.youtube.com/embed/L0RDrSKenGo&quot; frameborder=&quot;0&quot; allowfullscreen=&quot;&quot;&gt;&lt;/iframe&gt;

&lt;p&gt;&lt;br /&gt;
It is an interesting question: What happens if you mix &lt;em&gt;materials&lt;/em&gt; with code, symbols and data? In fact, what happens if you replace analog processes for fabrication of goods with digital abstractions to build things?&lt;/p&gt;

&lt;p&gt;As Neil Gershenfeld explains, these questions have led to the &lt;a href=&quot;https://youtu.be/L0RDrSKenGo?t=10m48s&quot;&gt;so-called fablabs or “Fabrication Labs”&lt;/a&gt;. In Fablabs, people use digital designs from the internet and transform them into physical objects. Often, 3D printers or laser cutters are used. It is amazing to see how blueprints and designs are moving freely over the internet. And, it is equally amazing to see how skills and know-how evolve if you visit a maker space regularly.&lt;/p&gt;

&lt;p&gt;If you have not yet been to a maker space before, here are some starting points. Some of the first FabLabs were built in Barcelona, where there was (is) a high unemployment rate among young people. The video below should give you an impression about a FabLab in Barcelona and about the people involved, &lt;a href=&quot;https://twitter.com/tomasdiez&quot;&gt;esp. Tomas Diez&lt;/a&gt;.&lt;/p&gt;

&lt;iframe width=&quot;560&quot; height=&quot;315&quot; src=&quot;https://www.youtube.com/embed/16Cu6nHrWyQ&quot; frameborder=&quot;0&quot; allowfullscreen=&quot;&quot;&gt;&lt;/iframe&gt;

&lt;p&gt;While not Barcelona, Munich is quite interesting for digital fabrication too. As I am in located in Munich, let’s look at some examples what “maker spaces” do and how they provide tools and parts.&lt;/p&gt;

&lt;h1 id=&quot;maker-spaces-in-munich&quot;&gt;Maker spaces in Munich&lt;/h1&gt;

&lt;p&gt;In Munich there are several maker spaces:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;There is &lt;a href=&quot;http://www.fablab-muenchen.de/&quot;&gt;FabLab Muenchen&lt;/a&gt; with many workshops for kids and a nice collection of 3D Printers.&lt;/li&gt;
  &lt;li&gt;Then there is &lt;a href=&quot;https://munichmakerlab.de/&quot;&gt;MunichMakerLab&lt;/a&gt; which has 3D Printers, a laser cutter, drones, and much more. It also hosts meetups for 3D Printing and Arduino for example.&lt;/li&gt;
  &lt;li&gt;Then there is the &lt;a href=&quot;http://hackaday.com/2014/11/17/because-you-cant-go-to-germany-without-seeing-model-trains/&quot;&gt;MCSM Model Engines Club&lt;/a&gt;. This is more of a shared workshop in essence, but it comes close to the idea of maker space. By the way, that space just moved into a building 200m away from the Intel HQ in Germany.&lt;/li&gt;
  &lt;li&gt;Last but not least, there is the &lt;a href=&quot;http://wiki.muc.ccc.de/mww:start&quot;&gt;Munich office of the Chaos Computer Club&lt;/a&gt; which comes close to the idea of maker space too.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In a maker space, you not only will find people to share ideas and solutions to problems. Often, you have access to materials and components for making things. For example, as I am working on a power supply for a robot with screws and cables that the MCSM space provides:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/maker_space.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Working with software, the skills and tools needed to build hardware are often easier to pick up in a maker space. 
While a maker space provides tools for analog and digital fabrication, you can explore the web for projects.&lt;/p&gt;

&lt;h1 id=&quot;blueprints-from-the-web&quot;&gt;Blueprints from the Web&lt;/h1&gt;

&lt;p&gt;Sharing hardware and circuits is still a bit different from sharing code. For example, you can easily share JavaScript libraries over &lt;a href=&quot;http://github.com&quot;&gt;Github&lt;/a&gt; or &lt;a href=&quot;http://npmjs.com&quot;&gt;NPM&lt;/a&gt;. On those platforms, you will also find plenty of code related to hardware. But where do you go for circuits or prototype hardware projects from others?&lt;/p&gt;

&lt;p&gt;For sharing blueprints of hardware prototypes, the &lt;a href=&quot;http://hackster.io&quot;&gt;Hackster.io&lt;/a&gt; platform is interesting. For example, take a look at this &lt;a href=&quot;https://www.hackster.io/ifoundthemeaningoflife/texting-automatic-cat-feeder-internet-o-tron-20076e&quot;&gt;automatic cat feeder&lt;/a&gt; project by &lt;a href=&quot;https://twitter.com/SelkeyMoonbeam&quot;&gt;@SelkeyMoonbeam&lt;/a&gt;. You will find an overview about used technologies, and most interestingly, you can access the parts inventory for the projects and sometimes a “toolbox”.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/supplies_hackster.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;When it comes to circuits, you can browse printed-circuit boards at &lt;a href=&quot;https://oshpark.com/shared_projects&quot;&gt;OSHpark&lt;/a&gt; for example. With a press of a button, you can easily share design efforts and explore a circuit from someone else. Sharing circuits is still in its early days.&lt;/p&gt;

&lt;p&gt;Another option to build circuits is by the &lt;a href=&quot;https://youtu.be/ZY9W-NIHMLA&quot;&gt;toner heat transfer method&lt;/a&gt;. With this, you print out the circuit on a plastic or paper and transfer it to a copper plate.&lt;/p&gt;

&lt;p&gt;For robots, Github is also an interesting platform for collaboration. For example, the &lt;a href=&quot;https://github.com/makenai/sumobot-jr&quot;&gt;Sumobot&lt;/a&gt; project allows you to build a small robot with an Arduino for less than 50 USD.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Exploring Noble and BlueIOT</title>
   <link href="https://thinkingonthinking.com/exploring-noble-and-blueiot/"/>
   <updated>2016-02-06T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/exploring-noble-and-blueiot</id>
   <content type="html">&lt;p&gt;(this is a draft)&lt;/p&gt;

&lt;p&gt;Imagine a &lt;a href=&quot;http://blog.telenor.io/iot/2015/08/04/smart-meetingroom.html&quot;&gt;smart meeting room&lt;/a&gt; that you can ping whether it is free or not.
Or, imagine a stroll in a museum where you can receive messages from artworks, connecting with the artist or other visitors.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Bluetooth_low_energy&quot;&gt;Bluetooth Low Energy&lt;/a&gt; might become an important building block to build these new kinds of connected experiences.&lt;/p&gt;

&lt;p&gt;And, as JavaScript developers, we are lucky that &lt;a href=&quot;https://github.com/sandeepmistry/&quot;&gt;Sandeep Mistry&lt;/a&gt; has translated much of the BLE specifications into Node.js modules.&lt;/p&gt;

&lt;p&gt;Based on his &lt;a href=&quot;https://github.com/sandeepmistry/noble&quot;&gt;Noble&lt;/a&gt; library, it is possible to learn what Bluetooth Low Energy does.&lt;/p&gt;

&lt;p&gt;But before coming to the JavaScript part, you need a peripheral, or a “beacon”. Bluetooth is easier to grasp with some test vehicles.&lt;/p&gt;

&lt;p&gt;This can be anything, but I go with the &lt;a href=&quot;https://www.tindie.com/products/FabLab/platinchen/&quot;&gt;BlueIOT&lt;/a&gt; module. BlueIOT is based on the &lt;a href=&quot;https://www.bluegiga.com/en-US/products/ble113-bluetooth-smart-module/&quot;&gt;BlueGiga BLE113&lt;/a&gt;. It has a couple of advantages compared to other solutions:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;BlueIOT has a very small form factor and can be powered from a coin cell battery, sleeping currents can be smaller than 1 uA&lt;/li&gt;
  &lt;li&gt;It uses a FCC/Bluetooth certified BLE module. This can save you time and costs if you want to develop your own Beacons later.&lt;/li&gt;
  &lt;li&gt;The module comes with an ATmega328p chip that can be easily programmed with the Arduino toolchain&lt;/li&gt;
  &lt;li&gt;It is possible to update the module with OTA and  connectivity configuration could be scripted with &lt;a href=&quot;http://ezoelectro.narod.ru/doc-pdf/ble112/BGScript_developer_guide_v2.5.pdf&quot;&gt;bgscript&lt;/a&gt;. This helps to separate application logic and communication logic, as well as reducing load on the host processor&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Besides the BlueIOT modules, there are a couple of alternatives to explore Bluetooth connectivity:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Michael Kroll has built a &lt;a href=&quot;https://github.com/michaelkroll/BLE-Shield&quot;&gt;BLE Arduino shield&lt;/a&gt; that you can buy from Seeedstudio&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;http://redbearlab.com/bleshield/&quot;&gt;RedbearLabs Arduino Shields and boards&lt;/a&gt;  are interesting if you like the Arduino form factor, and come with a number of nice apps and examples&lt;/li&gt;
  &lt;li&gt;The Intel Edison comes with Bluetooth built-in and you run the (Node.js) Bleno module to act as Beacon.&lt;/li&gt;
  &lt;li&gt;If you know a talented hardware hacker, you can also look at the Bluetooth module from &lt;a href=&quot;https://github.com/tessel/reach-hardware&quot;&gt;the Tessel project&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;h1 id=&quot;preparing-the-beacon&quot;&gt;Preparing the Beacon&lt;/h1&gt;

&lt;p&gt;To prepare working with the BlueIOT module, I build two little boards:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;The first board to support flashing a Beacond. It accepts a TTL-UART converter that fits well the Arduino toolchain (see &lt;a href=&quot;http://blog.farsinotare.com&quot;&gt;towards smaller devices&lt;/a&gt;)
&lt;img src=&quot;/static/images/blueiot_2.png&quot; /&gt;&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;The second board provides a LED and a coin cell to experiment with states of a beacond. It also has a jumper to save battery power.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/blueiot_1.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Besides the hardware helpers, you need to flash some firmware that can look as follows (credits to ]Guido Burger](https://twitter.com/guido_burger)):&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;  #include &amp;lt;Wire.h&amp;gt;
  #include &amp;lt;LowPower.h&amp;gt;
  #include &amp;lt;SPI.h&amp;gt;
  #include &amp;lt;SoftwareSerial.h&amp;gt;
  
  SoftwareSerial blueIOT(6, 9);
  
  void setup() {
    Serial.begin(9600);
     delay(1000);
     blueIOT.begin(57600);
     
     // BLE Module wakeup - HIGH prevents module to go to sleep mode
     pinMode(3, OUTPUT);
     digitalWrite(3, HIGH);
   
     // TEST LED on PIN5
     pinMode(5, OUTPUT);
     blueIOT.write(20); // BLE buffer size
     delay(1000);
     Serial.println(&quot;start&quot;);
  }
  
  void loop(){
    
    if (blueIOT.available()) {
      char c = blueIOT.read();
      Serial.println(c, HEX);
      
      if (c == &apos;5&apos;) {
        digitalWrite(5, HIGH);
        blueIOT.write(&quot;on&quot;);
      }
      else {
       digitalWrite(5, LOW);
       blueIOT.write(&quot;off&quot;);
      }
      blueIOT.write(&quot;idle&quot;);
    }
  }
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h1 id=&quot;using-ble-with-noblejs&quot;&gt;Using BLE with Noble.js&lt;/h1&gt;

&lt;p&gt;The first contact with Noble is probably via the &lt;a href=&quot;https://github.com/sandeepmistry/noble/blob/master/test.js&quot;&gt;beacon test script&lt;/a&gt; from the Noble repository.  The following steps are necessary:&lt;/p&gt;

&lt;p&gt;1) Your local Bluetooth “radio” (e.g. your laptop) must be powered on. This is not always the case, since many devices switch off Bluetooth to save power.
2) Once the “Radio” is running, you can start the “discovery” process
3) A discovered device is called “peripheral” and it provides “services”
4) You can “connect” to the peripheral to find out
5) You can then list the “services”
6) A service comes with “characteristics” that you can read or write to&lt;/p&gt;

&lt;p&gt;In the case of a BlueIOT beacon with a LED, the first step is discovering the device.&lt;/p&gt;

&lt;p&gt;Noble provides a number of events. So, on a first attempt to run this library, let’s try this:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var noble = require(&apos;noble&apos;);
var MyRadio = require(&apos;./ble_radio&apos;);

// the service unique identifier tells us that it&apos;s a BlueIOT beacon
var blueIOTServiceUuid = &apos;b4bdb9988f4a45f6a4076b48d79cfc2f&apos;;

// we mainly use the service to write the state of a LED
var blueIOTwriteCharacteristicUuid = &apos;b4bdb9988f4e45f6a4076b48d79cfc2f&apos;;

var myRadio = new MyRadio({serviceIds: [blueIOTServiceUuid]});

noble.on(&apos;stateChange&apos;, myRadio.startScanning);
noble.on(&apos;discover&apos;, myRadio.processDiscovery);
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;With this construction, the main events from the beacon are processed behind the scenes in the “BLERadio” module.&lt;/p&gt;

&lt;p&gt;Let’s look how to scan for devices and process “discoveries”.&lt;/p&gt;

&lt;h1 id=&quot;scanning-for-devices&quot;&gt;Scanning for devices&lt;/h1&gt;

&lt;p&gt;Scanning for devices is shown in the Noble examples:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;BLERadio.prototype.powerOn = function (state) {
  console.log(&apos;state change: &apos; + state);

  if (state === &apos;poweredOn&apos;) {

    // Once the BLE radio has been powered on, it is possible
    // to begin scanning for services. Pass an empty array to
    // scan for all services (uses more time and power).

    console.log(&apos;scanning...&apos;);

    if (this.serviceUuid) {
      noble.startScanning([this.serviceUuid], false);
    } else {
      noble.startScanning([], false);
    }
  }
  else {
    console.log(&apos;there is no BLE device. stop scanning.&apos;);
    noble.stopScanning();
  }
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;It is possible to scan only for specific devices. This is indicated with the serviceUuid.
Once the scanning has finished, you can process discoveries. I found the need to use a Connection module as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;BLERadio.prototype.processDiscovery = function(peripheral) {

  // once there is a peripheral, connect to it
  var connection = new Connection(peripheral);
  connection.start();

  noble.stopScanning();
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now, from a Connection you can interact with services and characteristics on the device.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>RIP Marvin Minsky</title>
   <link href="https://thinkingonthinking.com/marvin-minsky-passed-away/"/>
   <updated>2016-01-26T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/marvin-minsky-passed-away</id>
   <content type="html">&lt;p&gt;Much of engineering and programming is about solving puzzles, learning and making discoveries. That there are many emotions (like joy, fun, awe, pain, surprise) in thinking were one of the lessons prof. Minsky taught me. And, it was one of the ideas of starting my blog “&lt;a href=&quot;http://thinkingonthinking.com&quot;&gt;thinking_on thinking&lt;/a&gt;”.&lt;/p&gt;

&lt;iframe width=&quot;560&quot; height=&quot;315&quot; src=&quot;https://www.youtube.com/embed/-pb3z2w9gDg&quot; frameborder=&quot;0&quot; allowfullscreen=&quot;&quot;&gt;&lt;/iframe&gt;
&lt;p&gt;&lt;em&gt;The Society of Mind lecture by Prof. Marvin Minsky at MIT on OpenCourseWare&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;It was very sad news to hear that Marvin Minsky passed away. In honour to prof. Minsky, I want to share some email exchanges we had:&lt;/p&gt;

&lt;h2 id=&quot;2003&quot;&gt;2003&lt;/h2&gt;

&lt;p&gt;My first email to Marvin when I was studying semiconductors:&lt;/p&gt;

&lt;p&gt;Dear Prof. Minsky,&lt;/p&gt;

&lt;p&gt;lately I was thinking that computing systems can never
be “intelligent” (I haven’t thought about a definition
of intelligence yet, but I mean the function of
solving problems autonomously). As far as I understand
VLSI systems, they are all finite state machines.&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;Brain are finite-state machines, too.  So far as present-day physics can say.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;That kept me thinking and I had to ask more questions. I was wondering what the
evolutionary reason was, that humans have a certain
number of neuron cells.&lt;/p&gt;

&lt;p&gt;For example, an Intel Pentium processor has around 400 milion
transistors (90nm gate length). Moore’s Law is predicting several billions
transistors in some years. At the moment the biggest
bottleneck is seen in the power consumption (=heat
production). Do you know if there is a similar problem
for the human brain?&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;That is a very interesting question! In fact, many evolutionary 
biologists  think in terms of power consumption,  and regard the 
brain as so power-hungry that this is an important factor in natural 
selection;  they conclude that the reason why so few animals are 
intelligent,  is that it doesn’t have very much cost-benefit!&lt;/p&gt;
&lt;/blockquote&gt;

&lt;blockquote&gt;
  &lt;p&gt;Here is an amusing fact:  a typical human being consumes about 130 
watts of power.  All this power comes from pumping protons through 
the membranes of the mitochondria,  against a potential of about 300 
millivolts.  Therefore we are powered by a current of about 500 
amperes!&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;I am still busy in reading some of your work from your
homepage… I think there will many question popup.
Just one last remark, I have read your article about
&lt;a href=&quot;http://web.media.mit.edu/~minsky/papers/MusicMindMeaning.html&quot;&gt;music and meaning&lt;/a&gt;. I found there were extremely
interesting thoughts in it.&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;Thank you.    Many composers have told me that, so far as they know, 
that is the only good article specifically about theories of why 
people like music.    Isn’t that strange, considering that many 
people spend such a large fraction of their lives (and incomes) in it?&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;ps: I have recently read the book of Bertrand Russel
“Problems of Philosophy”. Do you have suggestions of
other books where perception of things/reality is
examined ?&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;Yes, very few philosophers have as much common sense as Russell did. 
The only ones I respect today are  Aaron Sloman,  Daniel Dennett, 
and John McCarthy.  ( Sloman and McCarthy have good Web pages.)  It 
is peculiar how few modern thinkers look so deeply into questions 
about how the mind really works.  But I still find important ideas 
in much older psychology books, especially Sigmund Freud, William 
James, Francis Galton,  Wilhelm Wundt, and some yet more ancient 
works by Aristotle and Augustine.    Needless to say, very few 
contemporary students or researchers have read any of these!&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2 id=&quot;2004&quot;&gt;2004&lt;/h2&gt;

&lt;p&gt;Then, I started reading more about symbol processing in our nervous system. Not sure if this message came via a newsgroup.&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;minsky@media.mit.edu schrieb:&lt;/p&gt;
&lt;/blockquote&gt;

&lt;blockquote&gt;
  &lt;p&gt;The power of consciousness comes not
from ceaseless change of state, but from having enough stability to
discern significant changes in your surroundings.  To “notice” change
requires the ability to resist it, in order to sense what persists
through time, but one can do this only by being able to examine and
compare descriptions from the recent past.  We notice change in spite
of change, and not because of it.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2 id=&quot;2006&quot;&gt;2006&lt;/h2&gt;

&lt;p&gt;Then, in 2006 I reflected about moving to Boston to study computers at the Media Lab. Still, moving from Germany to the US would be quite some jump for me at that time. Here is some resasoning of Marvin.&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;minsky@media.mit.edu schrieb:&lt;/p&gt;
&lt;/blockquote&gt;

&lt;blockquote&gt;
  &lt;p&gt;Well, it was not necessary to travel, because I was in the right place at the right time. When I was in college in the late 1940s, some of the greatest mathematicians in the world had already come to Harvard, as refugees from Europe and World War II. Then when I went to Princeton in 1950,  I had the opportunity to work with von Neumann and other great mathematicians, as well as physicists like Oppenheimer.  I did not get interested in logic until later, and never liked it much because I recognized that most reasoning involves analogies, which no logical systems can handle well.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;I also asked questions on music, what I was listening at that time:&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;I like Scarlatti’s ideas enough, but prefer to be guided by Beethoven and Bach, because they show how to think several thoughts at once.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;And about traveling in general:&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;As for visiting countries, I don’t much like cultures, because each of them only shows different mistakes.  To me, only the culture of Science shows progress.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2 id=&quot;2008&quot;&gt;2008&lt;/h2&gt;

&lt;p&gt;The last email conversation:&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;Marvin Minsky &lt;a href=&quot;mailto:minsky@media.mit.edu&quot;&gt;minsky@media.mit.edu&lt;/a&gt;
11/26/08 to me On Nov 12, 2008,&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;at 3:23 PM, Mulder Patrick wrote:&lt;/p&gt;

&lt;p&gt;I just discovered your writings at &lt;a href=&quot;http://wiki.laptop.org/go/The_OLPC_Wiki&quot;&gt;wiki.laptop.org&lt;/a&gt; on how education
could put incentives on discoveries for children.&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;Yes.  I’m right now trying to finish &lt;a href=&quot;http://web.media.mit.edu/~minsky/OLPC-5.html&quot;&gt;Memo 5&lt;/a&gt;, which is about whether we can &lt;em&gt;explicitly&lt;/em&gt; teach children new ways to think, etc.
I think I should copy those memos to my web page.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;I recently learnt about values within “teams” and how they
prevent on developing individual views:&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;Interesting issue: does teaching “cooperation” conflict with learning to develop one’s own ideas?&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;For example:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Jonathan Haidt &lt;a href=&quot;http://www.ted.com/index.php/talks/jonathan_haidt_on_the_moral_mind.html&quot;&gt;the moral mind&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;http://www.yourmorals.org/explore.php&quot;&gt;YourMorals&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;blockquote&gt;
  &lt;p&gt;I couldn’t quickly extract a clear idea from these. Does Jonathan Haidt have a good paper on this?&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;I did not understand too much how to dig for papers in social sciences at that time. But I liked to asked more questions about books in general. Also about writing.&lt;/p&gt;

&lt;p&gt;So I asked: It is not possible to regularly order your first book on computers.
Are there plans to re-publish the book? Do you have it as PDF or text?&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;Yes, I’ll try to republish it.  I encountered resistance, the last time I tried, because publishers insisted on something new.  But I wrote it so that it would never be outdated!&lt;/p&gt;
&lt;/blockquote&gt;

&lt;ul&gt;
  &lt;li&gt;I have your book on Perceptrons. Does topology play a role in modern
computers?  How can it teach us to make better abstractions?&lt;/li&gt;
&lt;/ul&gt;

&lt;blockquote&gt;
  &lt;p&gt;Well, to begin with, just deciding if there is more than one object in a visual scene need a topological process.&lt;br /&gt;
Or counting.  Or X is inside Y, etc.  Topology abstracts relationships that don’t depend on specific shapes.&lt;br /&gt;
In any case, I suspect that it would be a better subject for children than arithmetic is.  I have an age 10 granddaughter whose hobby is abstract category theory; I think she’s likely to become a powerful mathematician.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;blockquote&gt;
  &lt;p&gt;What’s interesting about that Perceptrons book is that, so far as I can see, 
no current neural-network practitioners understand its ideas at all!&lt;br /&gt;
So far as I can see, all of its conclusions still apply to multilayer networks.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;From these conversations, I learnt how important it can be to ask questions and how to search for new answers. These are the origins of my blog &lt;a href=&quot;http://thinkingonthiking.com&quot;&gt;thinkingonthinking&lt;/a&gt;.&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Experiences from Moore's Law</title>
   <link href="https://thinkingonthinking.com/hardware-cost-reductions/"/>
   <updated>2015-12-20T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/hardware-cost-reductions</id>
   <content type="html">&lt;p&gt;Hardware is getting cheaper and cheaper. It is fascinating to think about the origins of this phenomenon that people refer to &lt;a href=&quot;https://en.wikipedia.org/wiki/Moore%27s_law&quot;&gt;Moore’s Law&lt;/a&gt;. To understand how it works and where we are going, let’s look at computers under a microscope.&lt;/p&gt;

&lt;p&gt;On a physical level, computers are pieces of silicon with hundreds of millions (sometimes billions) of transistors. These transistors often act as switches, either to store state or to run binary operations. In contrast to mechanical switches, transistors act on voltages and electrical signals.&lt;/p&gt;

&lt;p&gt;Transistors are made from different layers of conducting, isolating and semi-conducting materials. All layers are added on top of a silicon substrate. Have a look at the picture below for a visual explanation of layers on a silicon wafer.&lt;/p&gt;

&lt;p&gt;&lt;br /&gt;&lt;/p&gt;
&lt;image src=&quot;/static/images/layers_silicon.png&quot; /&gt;

&lt;p&gt;&lt;br /&gt;&lt;/p&gt;

&lt;p&gt;Importantly, the shape of transistors is defined with help of &lt;a href=&quot;https://en.wikipedia.org/wiki/Photolithography&quot;&gt;photolithography&lt;/a&gt;. These machines that project images of transistors on a wafer are one of the main driving forces behind “Moore’s Law”. By using smaller wavelengths of light, it was possible to shrink transistors from micrometers in size to tens of nanometers. (It is generally believed that transistors can be shrunk to sub-nanometer size.)&lt;/p&gt;

&lt;p&gt;&lt;a data-flickr-embed=&quot;true&quot; href=&quot;https://www.flickr.com/photos/zeissmicro/9440302219/in/photolist-kUua-fod1Xn-5ZvEoJ-oWwhNq-oWwhJs&quot; title=&quot;Semiconductor Integrated Circuit (IC), BSE Image from ZEISS EVO&quot;&gt;&lt;img src=&quot;https://farm6.staticflickr.com/5323/9440302219_3c613ecbac_n.jpg&quot; width=&quot;320&quot; height=&quot;240&quot; alt=&quot;Semiconductor Integrated Circuit (IC), BSE Image from ZEISS EVO&quot; /&gt;&lt;/a&gt;&lt;script async=&quot;&quot; src=&quot;//embedr.flickr.com/assets/client-code.js&quot; charset=&quot;utf-8&quot;&gt;&lt;/script&gt;&lt;/p&gt;

&lt;p&gt;Besides going to smaller transistor sizes and higher system performance, Moore’s Law affects the costs of hardware devices. The more transistors you can put on a wafer, the cheaper manufacturing of single device becomes. This is similar to buying a burger in McDonalds, the higher their production volume, the less costly a single unit becomes.&lt;/p&gt;

&lt;p&gt;Now, while the costs of a chip are heavily influenced by the size of a transistor, many more things happen until a chip is sold. Especially packaging of a chip is increasingly important. In many Maker projects, it’s nice to have large packages for easier handling. But with increasing size, device warehousing and transports mean higher costs than smaller packages. So, while Moore’s Law fits very well to explain cost reduction of individual chips, cost reductions of packages and systems might have different origins. Cheap manufacturing in &lt;a href=&quot;https://www.reddit.com/r/arduino/comments/3vswl4/arduino_clones_for_5_usd_how_does_it_work_to_get/&quot;&gt;China&lt;/a&gt; for example is one reason that cheap Arduino boards exists.&lt;/p&gt;

&lt;p&gt;Note: The role of Moore’s Law is important in developing &lt;a href=&quot;http://embeddednodejs.com&quot;&gt;embedded devices for the Internet of Things&lt;/a&gt;.&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Espruino - JavaScript on bare metal</title>
   <link href="https://thinkingonthinking.com/espruino-javascript-on-bare-metal/"/>
   <updated>2015-11-27T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/espruino-javascript-on-bare-metal</id>
   <content type="html">&lt;p&gt;Probably the easiest way to get started with JavaScript and embedded devices are Espruino boards. Espruino is “mini” Node.js for embedded devices.&lt;/p&gt;

&lt;p&gt;The &lt;a href=&quot;http://espruino.com&quot;&gt;Espruino project&lt;/a&gt; by Gordon Williams was &lt;a href=&quot;https://www.kickstarter.com/projects/gfw/espruino-javascript-for-things&quot;&gt;funded in a Kickstarter campaign 2013&lt;/a&gt; and &lt;a href=&quot;http://makezine.com/2014/11/11/javascript-by-the-pico/&quot;&gt;has been continuously improved since&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/espruino.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;A simple blinking LED example with Espruino can look as follows. First, you mount an &lt;a href=&quot;http://www.espruino.com/Pico&quot;&gt;Espruino Pico&lt;/a&gt; (which costs around 30USD) with a LED and resistor on a breadboard as is shown above. In this case, I use pin B7 and GND as shown in detail on the &lt;a href=&quot;http://www.espruino.com/Pico&quot;&gt;hardware pinout&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;In contrast to an Arduino Uno or other microcontrollers, you don’t need to setup an IDE to run code on Espruino. On most systems, you don’t even need to install install drivers. You only need a terminal program which comes pre-installed on many computers these days.&lt;/p&gt;

&lt;p&gt;Once the hardware is setup, you can connect to the Espruino via USB and a terminal program. A good terminal program for MacOS and Linux is “screen”, while “hterm” is a popular choice on Windows machines.&lt;/p&gt;

&lt;p&gt;On MacOS, you can connect to the Espruino with ‘screen’ as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ screen /dev/cu.usbmodem141 9600
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Once you are connected you should a welcome screen similar to the one below.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/espruino_welcome.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;You now can turn the LED on B7 ON with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; digitalWrite(B7, HIGH);
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And OFF, with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; digitalWrite(B7, LOW);
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;You could add a toggle function to the Espruino workspace with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; var pinStates=[];
&amp;gt; function togglePin(pin) {
    pinStates[pin] = !pinStates[pin];
    digitalWrite(pin, pinStates[pin]);
  }
&amp;gt; togglePin(B7);  // ON
&amp;gt; togglePin(B7);  // OFF
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Espruino comes with some more &lt;a href=&quot;http://www.espruino.com/Reference#_global&quot;&gt;global functions&lt;/a&gt; built-in.&lt;/p&gt;

&lt;p&gt;For example, you could attach (or “bind”) functions to a certain pin, e.g. with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; toggleB7 = togglePin.bind(null, B7)
&amp;gt; toggleB7();
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And to toggle the LED off, use:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; toggleB7();
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Besides digital signals, an Espurino makes working with analog signals equally simple with &lt;a href=&quot;http://www.espruino.com/Reference#l__global_analogWrite&quot;&gt;analogWrite&lt;/a&gt;. For example, a pulse-width-modulated (PWM) signal can be simply be generated with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; analogWrite(B7, 0.3, {freq: 1000});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Or, to generate flickering pulses of 10 Herz would be possible with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; analogWrite(B7, 0.5, {freq: 10})
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;In contrast to Raspberry Pi and other Linux based devices, Espruino fills the gap for programming battery powered devices with JavaScript.&lt;/p&gt;

&lt;p&gt;Boards can be obtained from &lt;a href=&quot;http://espruino.com&quot;&gt;Espruino.com&lt;/a&gt;. Or, if you have some extra time and/or interests in deeper embedded software development, you can flash the JavaScript firmware to other boards, such as STM32 based discovery boards. Other boards are listed &lt;a href=&quot;http://www.espruino.com/Other+Boards&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;New Espruino devices even support connectivity with Wifi or Ethernet.&lt;/p&gt;

&lt;h2 id=&quot;update-espruino-on-esp8266&quot;&gt;Update: Espruino on ESP8266&lt;/h2&gt;

&lt;p&gt;Thanks to the comment of &lt;a href=&quot;https://github.com/0xPIT/&quot;&gt;0xPIT&lt;/a&gt;, I played a bit with Espruino and an ESP8266. The board I had was an ESP8266 12 from Tronixlabs. It has a jumper to enter bootmode (GPIO0).&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/espruino_esp8266.jpg&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Now, to flash Esupruino firmware to the ESP8266, I first cloned this &lt;a href=&quot;https://github.com/aplikatika/Espruino-on-ESP8266&quot;&gt;Espruino port for ESP8266&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;The flashing process of an ESP8266I is nicely explaines in &lt;a href=&quot;http://williamdurand.fr/2015/03/17/playing-with-a-esp8266-wifi-module/&quot;&gt;this guide&lt;/a&gt;. The main part is getting a copy of “esptool.py”.&lt;/p&gt;

&lt;p&gt;Then, flashing the Espruino firmware with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ esptool.py --port /dev/cu.usbserial-FT912ZMF write_flash 0x00000 firmware/0x00000.bin 0x10000 firmware/0x10000.bin
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And, last flashing a JavaScript script onto the ESP8266 with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ esptool.py --port /dev/cu.usbserial-FT912ZMF write_flash 0x60000 ./scripts/hello.js
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;When you then connect to the board with an FTDI-to-USB cable, you’ll see that the script is executed:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/esp_boot.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;(Note: if you are new to electronics: Breadboards and components will be discussed in my new book &lt;a href=&quot;http://embeddednodejs.com&quot;&gt;Node.js for embedded systems&lt;/a&gt;).&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>All the World's Futures</title>
   <link href="https://thinkingonthinking.com/end-of-industrialisation/"/>
   <updated>2015-10-02T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/end-of-industrialisation</id>
   <content type="html">&lt;p&gt;No doubt, the world is changing. In a world with global supply-chain networks, computers and robots, how valid are theories of division of labor? In a world, that is shaped by Utilitarianism, what value can arts and aesthetics add?&lt;/p&gt;

&lt;p&gt;These questions came to mind when entering the Venice Biennale 2015 curated by Okui Enwezor. Before entering the exhibition, we are reminded: “Blues Blood Bruise”.&lt;/p&gt;

&lt;p&gt;&lt;a data-flickr-embed=&quot;true&quot; href=&quot;https://www.flickr.com/photos/pmulder99/21884104192/in/dateposted-public/&quot; title=&quot;DSCF2425&quot;&gt;&lt;img src=&quot;https://farm1.staticflickr.com/692/21884104192_a4cd81e418_n.jpg&quot; width=&quot;320&quot; height=&quot;240&quot; alt=&quot;DSCF2425&quot; /&gt;&lt;/a&gt;&lt;script async=&quot;&quot; src=&quot;//embedr.flickr.com/assets/client-code.js&quot; charset=&quot;utf-8&quot;&gt;&lt;/script&gt;&lt;/p&gt;

&lt;p&gt;Exploiting humans as slaves was common before the first industrial revolution (and still is today in some countries). In contrast to machines, humans can “feel” work.&lt;/p&gt;

&lt;p&gt;&lt;a data-flickr-embed=&quot;true&quot; href=&quot;https://www.flickr.com/photos/pmulder99/21758045279/in/dateposted-public/&quot; title=&quot;DSCF2562&quot;&gt;&lt;img src=&quot;https://farm1.staticflickr.com/622/21758045279_b5fa970dbc_n.jpg&quot; width=&quot;320&quot; height=&quot;240&quot; alt=&quot;DSCF2562&quot; /&gt;&lt;/a&gt;&lt;script async=&quot;&quot; src=&quot;//embedr.flickr.com/assets/client-code.js&quot; charset=&quot;utf-8&quot;&gt;&lt;/script&gt;&lt;/p&gt;

&lt;p&gt;Slavery was not only a chapter in the dark ages of colonial times. In the last century, doctrines of fascism resulted into slavery too. In these times, human rights, ethics and aesthetics were lost and some humans declared themselves as “Uber-Mensch”.&lt;/p&gt;

&lt;p&gt;&lt;a data-flickr-embed=&quot;true&quot; href=&quot;https://www.flickr.com/photos/pmulder99/21757076038/in/dateposted-public/&quot; title=&quot;DSCF2558&quot;&gt;&lt;img src=&quot;https://farm6.staticflickr.com/5643/21757076038_4c2187eda2_n.jpg&quot; width=&quot;320&quot; height=&quot;240&quot; alt=&quot;DSCF2558&quot; /&gt;&lt;/a&gt;&lt;script async=&quot;&quot; src=&quot;//embedr.flickr.com/assets/client-code.js&quot; charset=&quot;utf-8&quot;&gt;&lt;/script&gt;&lt;/p&gt;

&lt;p&gt;In a free world, we believe that “hard” work can make us better off. Humans become refugees in the hope of a better life and work elsewhere.&lt;/p&gt;

&lt;p&gt;&lt;a data-flickr-embed=&quot;true&quot; href=&quot;https://www.flickr.com/photos/pmulder99/21757080498/in/dateposted-public/&quot; title=&quot;DSCF2560&quot;&gt;&lt;img src=&quot;https://farm1.staticflickr.com/773/21757080498_e99fe740f4_n.jpg&quot; width=&quot;320&quot; height=&quot;240&quot; alt=&quot;DSCF2560&quot; /&gt;&lt;/a&gt;&lt;script async=&quot;&quot; src=&quot;//embedr.flickr.com/assets/client-code.js&quot; charset=&quot;utf-8&quot;&gt;&lt;/script&gt;&lt;/p&gt;

&lt;p&gt;Economic immigration in our times will be one of the many challenges we are facing. For example, the Mediterranean sea that once was flourishing in trade and mixing cultures, has become a border from first world to 3rd world.&lt;/p&gt;

&lt;p&gt;&lt;a data-flickr-embed=&quot;true&quot; href=&quot;https://www.flickr.com/photos/pmulder99/21759076219/in/dateposted-public/&quot; title=&quot;DSCF2573&quot;&gt;&lt;img src=&quot;https://farm1.staticflickr.com/610/21759076219_ec51909178_n.jpg&quot; width=&quot;320&quot; height=&quot;240&quot; alt=&quot;DSCF2573&quot; /&gt;&lt;/a&gt;&lt;script async=&quot;&quot; src=&quot;//embedr.flickr.com/assets/client-code.js&quot; charset=&quot;utf-8&quot;&gt;&lt;/script&gt;&lt;/p&gt;

&lt;p&gt;The hidden forces of capitalism drives people to risk their lives in search for a “better” life.  Looking at “All the World’s Futures”, the “laws” of capitalism might have deeper roots in human nature, than economic theories might suggest.&lt;/p&gt;

&lt;p&gt;&lt;a data-flickr-embed=&quot;true&quot; href=&quot;https://www.flickr.com/photos/pmulder99/21708240008/in/dateposted-public/&quot; title=&quot;DSCF2419&quot;&gt;&lt;img src=&quot;https://farm1.staticflickr.com/715/21708240008_bdd4f4d88e_n.jpg&quot; width=&quot;320&quot; height=&quot;240&quot; alt=&quot;DSCF2419&quot; /&gt;&lt;/a&gt;&lt;script async=&quot;&quot; src=&quot;//embedr.flickr.com/assets/client-code.js&quot; charset=&quot;utf-8&quot;&gt;&lt;/script&gt;&lt;/p&gt;

&lt;p&gt;With Internet and computers, we stop having a “western” culture based on mass production. In this world to come, what would you see? What identity would you have? And, how would you “identify” yourself and friends? This is where the arts can help. Let’s look again at “All the World’s Futures” and work on the best ones.&lt;/p&gt;

&lt;p&gt;&lt;a data-flickr-embed=&quot;true&quot; href=&quot;https://www.flickr.com/photos/pmulder99/21758067519/in/dateposted-public/&quot; title=&quot;DSCF2567&quot;&gt;&lt;img src=&quot;https://farm6.staticflickr.com/5763/21758067519_c33121156c_n.jpg&quot; width=&quot;320&quot; height=&quot;240&quot; alt=&quot;DSCF2567&quot; /&gt;&lt;/a&gt;&lt;script async=&quot;&quot; src=&quot;//embedr.flickr.com/assets/client-code.js&quot; charset=&quot;utf-8&quot;&gt;&lt;/script&gt;&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Booting single board computers</title>
   <link href="https://thinkingonthinking.com/single-board-computers/"/>
   <updated>2015-08-03T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/single-board-computers</id>
   <content type="html">&lt;p&gt;Booting processes of computers are opaque. When a board boots, it setups internal resources and peripherals. One such peripheral can be a display. And surprisingly, there is not much documantation out there about what is important.&lt;/p&gt;

&lt;h2 id=&quot;raspberry-pi&quot;&gt;Raspberry Pi&lt;/h2&gt;

&lt;p&gt;Some examples can be found in talking to a display on an Raspberry Pi. First, some reading about &lt;a href=&quot;https://en.wikipedia.org/wiki/Linux_framebuffer&quot;&gt;“framebuffers”&lt;/a&gt; is needed. To use a display on the Raspberry Pi, you must build a framebuffer device first. There is a nice &lt;a href=&quot;https://github.com/notro/fbtft/wiki&quot;&gt;write-up&lt;/a&gt; here, and I arrived quickly at a working device:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;/dev/fb1
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;To then show an image on the display requires a command “fbi”. First, we must install the program:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# apt-get install fbi
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Then &lt;a href=&quot;https://www.raspberrypi.org/forums/viewtopic.php?f=27&amp;amp;t=15182&quot;&gt;you could see the image quickly by running&lt;/a&gt;:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# fbi -T 2 IMAGE.jpg
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;I played with some scripts to change the image, but the processor on the RPi B was not fast enough (or my images were too large) to make it really fun.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/rpi_display.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;In my second project this week, I explored a new Linux image for the Intel Galileo. This was harder than expected.&lt;/p&gt;

&lt;p&gt;My main idea is: Build an SD Card with Node.js and some libraries pre-installed. Following the manuals, I formatted the SD card with an FAT32 filesystem. But booting that one was difficult. Formatting the card with an ext3 filesystem made the content readable, but the SD Card was neither bootable.&lt;/p&gt;

&lt;p&gt;Exploring this problem, let me to play with the &lt;a href=&quot;http://www.dedoimedo.com/computers/grub.html#mozTocId616834&quot;&gt;Grub bootloader&lt;/a&gt; and &lt;a href=&quot;http://www.mepis.org/docs/en/index.php?title=GRUB_from_command_line&quot;&gt;Grub from the command line&lt;/a&gt;. Also, &lt;a href=&quot;https://en.wikipedia.org/wiki/Unified_Extensible_Firmware_Interface&quot;&gt;UEFI&lt;/a&gt; might open new doors and I discvered &lt;a href=&quot;http://wiki.osdev.org/Main_Page&quot;&gt;a wiki dedicated to development of operating systems&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Why bother? We usually pay money for this (to Apple or Microsoft) because we have better things to do, then learing about the boot process of an OS. But if we want to build embedded devices that have constrained resources, understanding an operating system becomes important again.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>On Innovation Communities</title>
   <link href="https://thinkingonthinking.com/on-innovation-communities/"/>
   <updated>2015-06-30T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/on-innovation-communities</id>
   <content type="html">&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Democratization&quot;&gt;Democratization&lt;/a&gt; of hardware products was an important topic at this year’s &lt;a href=&quot;http://solidcon.com/&quot;&gt;O’Reilly Solid conference&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;In his keynote, &lt;a href=&quot;https://twitter.com/joi&quot;&gt;Joi Ito&lt;/a&gt; was the first to mention, that open-source technologies provide a low-cost approach to innovation. Unlike “MBA driven innovation”, where money comes first, open-source technologies pushes innovation power to the bottom. Facebook, Youtube, and many Kickstarter projects were built with this approach.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/jo_ito.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;In this approach, modularity plays an important role as you don’t have to re-invent every piece from scratch. He pointed to Shenzen where kids are able to develop smartphones and watches as fast as we develop apps and websites. As the costs for innovation decrease, Ito provided foundations to extend Moore’s Law into biology. With “cheap” DNA sequencers, it might be possible to store and process very dense information in proteins and cells one day. This might revolutionize healthcare and biology, but also information technologies.&lt;/p&gt;

&lt;p&gt;The talk “Demoaterializing Mannufacturing” by &lt;a href=&quot;https://twitter.com/czingerkevin&quot;&gt;Kevin Czinger&lt;/a&gt; was especially visionary. Today, the debate is about the type of powertrain a car uses (fossil vs. electrical). However, looking at CO2 emissions, material costs and manufacturing, the debate should really be about &lt;em&gt;manufacturing&lt;/em&gt; of cars.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/fuel_costs.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Instead of building cars in large volumes, Czinger argued that an Ikea approach to car production would make more sense. A car could be built with simple modules: “Carduino”.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/carduino.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Besides less ecological impact of these cars, the costs of microfactories for 3D printed cars could be much lower than the costs for classical manufacturing.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/capital_light.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;By the way, these ideas are not science-fiction but real as you can see from the 3D printed car below.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/carduino_car.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Then, the impact of new technologies such as 3D Printing and Fablabs were discussed in a talk by TechShop founder Mark Hatch.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/maker_movment_2.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/maker_movement.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The impact of “open” skills and education on the job market was also discussed in an interview of Megan Smith from the US government. Smith mentioned the book &lt;a href=&quot;http://www.amazon.com/The-Geography-Jobs-Enrico-Moretti/dp/0544028058&quot;&gt;“The New Geopgrahy of Jobs”&lt;/a&gt; that shows how innovation not only happens in Silicon Valley and other technology hubs. She gave concrete examples where people could escape poverty by learning programming or maker skills.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/geo_jobs.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;An interesting talk on innovation &lt;em&gt;commnities&lt;/em&gt; was given by Edwina Portocarrero. In her talk, “Thinking it here, doing it there”, she showed how “citizen talent” plays a vital role in changing the “broken” face of Mexico City. Her message: You must invite people to co-create. She finished her talk with a quote by Jane Jacobs: “Cities have the capability of providing something to everybody, only because and only when, they are created by everybody.”&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/playgrounds_talk.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;If you want to experience more of these ideas, keep an eye on &lt;a href=&quot;http://solidcon.com/internet-of-things-eu-2015&quot;&gt;SolidCon in Amsterdam&lt;/a&gt; or SolidCon 2016. If you want to learn more about Democratization of Innovation, the books by Eric von Hippel are &lt;a href=&quot;http://evhippel.mit.edu/books/&quot;&gt;free for download&lt;/a&gt; (also at Amazon). A number of slidedecks from SolidCon can be found &lt;a href=&quot;http://solidcon.com/slides&quot;&gt;here&lt;/a&gt;. Also interesting is the talk by Massimo Banzi at the Computer Museum, where he explains the &lt;a href=&quot;https://blog.arduino.cc/2015/06/16/watch-massimo-presentation-at-computer-history-museum/&quot;&gt;history of Arduino&lt;/a&gt;. Would be great to see him giving a keynote at SolidCon 2016.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Push State from an Embedded Device</title>
   <link href="https://thinkingonthinking.com/push-state-from-an-embedded-device/"/>
   <updated>2015-06-14T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/push-state-from-an-embedded-device</id>
   <content type="html">&lt;p&gt;In this post, I want to show you how to read sensor data and work with it in the browser. To solve this problem, you need to “push” state from an embedded device into a &lt;a href=&quot;https://en.wikipedia.org/wiki/Network_socket&quot;&gt;“socket”&lt;/a&gt;. For the web, this usually means applying the &lt;a href=&quot;https://en.wikipedia.org/wiki/WebSocket&quot;&gt;WebSocket protocol&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/epoch_analog.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;We will use what we have learned in the previous &lt;a href=&quot;http://thinkingonthinking.com/serial-communication-with-nodejs/&quot;&gt;post on serial communication with Node.js&lt;/a&gt;. If you haven’t read that post, you can also run the embedded &lt;a href=&quot;http://yeoman.io/&quot;&gt;Yeoman&lt;/a&gt; generator which provides a number of files to get started. If you have read that post, you probably can skip the first section on getting the project files.&lt;/p&gt;

&lt;p&gt;The hardware used is an Arduino with the Grove Kit shield as shown below.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/grove_kit.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Furthermore, a simple LED and an input switch are nice for debugging the hardware setup. The LED and input button are shown below:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/grove_kit_led.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;And the button:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/grove_kit_btn.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;With the code from this post, you will be able to start a socket connection on client and server, and move data from an Arduino with serial communication.&lt;/p&gt;

&lt;h1 id=&quot;a-basic-web-server&quot;&gt;A basic web server&lt;/h1&gt;

&lt;p&gt;Assuming you want to start with a fresh Node.js project, you need to install some basic dependencies and setup some files. Yeoman provides many “generators” for different type of projects. My &lt;a href=&quot;https://github.com/mulderp/generator-embedded&quot;&gt;embedded Yeoman generator&lt;/a&gt; provides some help with a basic webserver that has a Firmata connection for Arduino.&lt;/p&gt;

&lt;p&gt;First, you must install Yeoman and the generator with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ npm install -g yo
$ npm install -g generator-embedded
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now, you can run:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ yo embedded
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;As first check, we can run the server with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ node server
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This should result into something similar to:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;Starting server at port: 3474
There is a problem with: /dev/cu.usbmodem14231
[Error: Cannot open /dev/cu.usbmodem14231]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This error appears because we not yet have an Arduino connected. Let’s change this.&lt;/p&gt;

&lt;h1 id=&quot;arduino-setup&quot;&gt;Arduino Setup&lt;/h1&gt;

&lt;p&gt;It is often a good idea to read out the position of a potentiometer before reading data from a sensor. The potentiometer I use is shown below from the Grove Kit.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/grove_kit_pot.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;A potentiometer behaves somewhat similar to a sensor. In both cases, a continuous signal (= “analog” quantity) is sampled by the microcontroller and converted into a digital quantity which a computer can work with it.&lt;/p&gt;

&lt;p&gt;In your project directory, you will find the following project files:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;package.json
server.js
static/index.html
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Let’s first look into the &lt;em&gt;package.json&lt;/em&gt;. The interesting part are the dependencies:&lt;/p&gt;

&lt;p&gt;“dependencies”: {
    “firmata”: “^0.4.1”,
    “ws”: “^0.7.2”,
    “finalhandler”: “^0.3.5”,
    “morgan”: “^1.5.2”,
    “router”: “^1.1.0
  }&lt;/p&gt;

&lt;p&gt;These are the minimum libraries to get a working server supporting both, an Arduino with Firmata and WebSockets. Let’s run:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ npm install
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Next, look through the following code for “server.js” to see what we are building. You will see how it will operate and interacts with the Arduino in a second.&lt;/p&gt;

&lt;p&gt;server.js&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var fs = require(&apos;fs&apos;);

// serial port with Firmata
var firmata = require(&apos;firmata&apos;);

// reference to embedded device
var modem = &apos;/dev/cu.usbmodem14231&apos;;

// setup web server
var port = 3474;
var Router = require(&apos;router&apos;);
var router = Router();
var http = require(&apos;http&apos;);
var finalhandler = require(&apos;finalhandler&apos;);
var morgan = require(&apos;morgan&apos;);
console.log(&apos;Starting server at port: &apos; + port);

// HTTP logging
router.use(morgan(&apos;default&apos;));

// sockets to push bytes
var WebSocketServer = require(&apos;ws&apos;).Server;

// basic static files
router.get(&apos;/&apos;, function(req, res) {
    res.writeHead(200, {&apos;Content-Type&apos;: &apos;text/html&apos;});
    res.end(fs.readFileSync(&apos;static/index.html&apos;));
});
router.get(&apos;/bundle.js&apos;, function(req,res) {
    res.writeHead(200, {&apos;Content-Type&apos;: &apos;application/javascript&apos;});
    res.end(fs.readFileSync(&apos;static/bundle.js&apos;));
});

// API to hardware
var api = Router();

api.get(&apos;/:pin/check&apos;, function(req, res) {
  board.digitalWrite(req.params.pin, 1);
  res.writeHead(200, {&apos;Content-Type&apos;: &apos;application/json&apos;});
  res.end(JSON.stringify({status: &apos;ok&apos;}));
});
router.use(&apos;/api&apos;, api);

// board
var board = new firmata.Board(modem, function(err){
  if (!err) {
    console.log(&apos;connected: &apos; + modem);
  } else {
    console.log(&apos;problem with: &apos; + modem);
    console.log(err);
    process.exit()
  }

  // configure ports
  board.pinMode(5, board.MODES.INPUT);

  // prepare server
  var server = http.createServer(function (req, res) {
    router(req, res, finalhandler(req, res));
  }).listen(port);

  // add websockets
  var wss = new WebSocketServer({server: server});
  wss.on(&apos;connection&apos;, function connection(ws) {
    console.log(&apos;websocket connected&apos;);
    ws.on(&apos;message&apos;, function incoming(message) {
      console.log(&apos;received: %s&apos;, message);
    });

    board.digitalRead(5, function(val, err) {
      if (val == 1) {
        console.log(val);
        ws.send(&apos;{&quot;state&quot;: &apos; + val + &apos;}&apos;);
      }
    });
  });
});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The first part of this server provides a bridge from HTTP to an Arduino. Then, the Arduino board is connected with help of the &lt;a href=&quot;http://www.arduino.cc/en/Reference/Firmata&quot;&gt;Firmata Library&lt;/a&gt;.  This is a common setup for controlling things on the embedded device.&lt;/p&gt;

&lt;p&gt;Let’s do a quick check from the command line by getting an LED to blink:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; $ curl http://localhost:3474/api/6/check
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;If you have put an LED to pin 6, you should now see it ON. But immediately after issueing the HTTP request, the connection between client and server is closed. If we want to stream sensor data from an Arduino, we need to look at a different approach.&lt;/p&gt;

&lt;h1 id=&quot;push-data-with-websockets&quot;&gt;Push Data with WebSockets&lt;/h1&gt;

&lt;p&gt;For many cases, we want an open connection between Arduino and browser to process measurements from sensors in realtime. This approach is given with WebSockets. A socket is a virtual communication device that can be shared between different processes on a computer. A WebSocket provides a protocol to work with sockets in a network.&lt;/p&gt;

&lt;p&gt;In the setup from above, the &lt;a href=&quot;https://www.npmjs.com/package/ws&quot;&gt;ws library&lt;/a&gt; for WebSockets in Node.js is already setup. For browsers, a [WebSocket API](https://developer.mozilla.org/en-US/docs/WebSockets/Writing_WebSocket_client_applications#Availability_of_WebSockets&lt;/p&gt;

&lt;p&gt;) is often included natively.&lt;/p&gt;

&lt;p&gt;To test WebSockets from the command line, you can install a nice command line tool wscat:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ npm install -g wscat
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now, if you start the server:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ node server
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And run:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ wscat -c ws://localhost:3474
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;You can already watch inputs from a digital switch on digital input 6. This should look similar to:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;lt; {&quot;state&quot;: 1}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now, take the potentiometer and connect it to e.g. analog input 5.&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;board.analogRead(3, function(val, err) {
    console.log(val);
    ws.send(&apos;{&quot;analog&quot;: &apos; + val + &apos;}&apos;);
});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Let’s check again with wscat:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ wscat -c ws://localhost:3474
&amp;lt; {&quot;analog&quot;: 643}
&amp;lt; {&quot;analog&quot;: 643}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;You now can observe the websocket connection in the browser and update a chart. This work is done in index.html&lt;/p&gt;

&lt;p&gt;static/index.html&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;lt;script&amp;gt;
  var ws = new WebSocket(&apos;ws://localhost:3474&apos;);

  ws.onmessage = function(e) {
      var timestamp = ((new Date()).getTime() / 1000)|0;
      var data = JSON.parse(e.data);
      var dataPoint = {time:  timestamp, y: data.analog};
      console.log(dataPoint);
  };
&amp;lt;/script&amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;A full example with plot function is &lt;a href=&quot;https://github.com/mulderp/nodejs-websockets-template&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;h1 id=&quot;wrap-up&quot;&gt;Wrap up&lt;/h1&gt;

&lt;p&gt;This post provided an outline to read data from an embedded device with Node.js and WebSockets. First, you had some contact with the web server. Then, the data was fed with a web socket into a browser. We still miss some basic connection to provide data over a network with a publish-subscribe pattern.&lt;/p&gt;

&lt;h1 id=&quot;references&quot;&gt;References&lt;/h1&gt;

&lt;p&gt;Maybe you want to plot it similar to &lt;a href=&quot;http://www.instructables.com/id/Plotting-Data-From-Arduino/&quot;&gt;plotting data from an Arduino&lt;/a&gt;.&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Some thoughts about writing</title>
   <link href="https://thinkingonthinking.com/some-thoughts-about-writing/"/>
   <updated>2015-05-25T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/some-thoughts-about-writing</id>
   <content type="html">&lt;p&gt;After writing &lt;a href=&quot;http://pipefishbook.com&quot;&gt;The Pipefishbook&lt;/a&gt;, there were some book reviews complaining: “This is not a full stack book.” “This book does not go deep enough.” “This book is more about automation than it is about web applications”.&lt;/p&gt;

&lt;p&gt;True, the book tries to discuss many topics. And, the directions of the book changed two times. First, the initial idea of the book was about writing on &lt;a href=&quot;http://thinkingonthinking.com/writing-on-ruby-and-backbone/&quot;&gt;combining Ruby-on-Rails with Backbone.js&lt;/a&gt; in the beginning of 2013. At that time, the idea of a “full stack” engineer was still vague. You either worked on the backend or the frontend, but you would not combine both worlds.&lt;/p&gt;

&lt;p&gt;Now, technical writing certainly is different from writing prose. And it is hard to find other technical writers sharing thoughts about writing. So, while trying to improve, I looked at the writing process from prose authors the last weeks.&lt;/p&gt;

&lt;p&gt;For many, writing starts with &lt;em&gt;not&lt;/em&gt; knowing, as John Drufesne explains in &lt;a href=&quot;https://www.youtube.com/watch?v=urJDbQl5W0I&amp;amp;feature=youtu.be&amp;amp;t=1m18s&quot;&gt;this TED talk&lt;/a&gt;. He  tells us: “What you don’t know is more important than what you know, because, what you don’t know is what engages your senses of wonder.”&lt;/p&gt;

&lt;p&gt;In fact, cultivating this sense of wonder is extremely important for learning and writing. Isaac Asimov explains it &lt;a href=&quot;https://www.youtube.com/watch?v=1CwUuU6C4pk&amp;amp;feature=youtu.be&amp;amp;t=4m53s&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;I guess dealing with failure and not knowing is part of the writing process. But, it is also a kind of home that you can build for yourself and some readers hopefully. It is an inspiring idea from Elizabeth Gilbert, and the way she puts it in her TED talk is well worth watching in: “Success, failure, and the drive to keep creating.”&lt;/p&gt;

&lt;iframe src=&quot;https://embed-ssl.ted.com/talks/elizabeth_gilbert_success_failure_and_the_drive_to_keep_creating.html&quot; width=&quot;640&quot; height=&quot;360&quot; frameborder=&quot;0&quot; scrolling=&quot;no&quot; webkitallowfullscreen=&quot;&quot; mozallowfullscreen=&quot;&quot; allowfullscreen=&quot;&quot;&gt;&lt;/iframe&gt;
</content>
 </entry>
 
 <entry>
   <title>Building an Arduino sketch from scratch</title>
   <link href="https://thinkingonthinking.com/an-arduino-sketch-from-scratch/"/>
   <updated>2015-03-31T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/an-arduino-sketch-from-scratch</id>
   <content type="html">&lt;p&gt;Unix programmers have been building software for many years with the command line and the &lt;a href=&quot;http://en.wikipedia.org/wiki/GNU_Compiler_Collection&quot;&gt;GCC toolchain&lt;/a&gt;. When developing for embedded systems, where clock cycles and memory usage counts, some familiarity with GCC is often needed.&lt;/p&gt;

&lt;h1 id=&quot;the-gcc-toolchain&quot;&gt;The GCC toolchain&lt;/h1&gt;

&lt;p&gt;The GCC tools come in different forms. For example, there is a variant for &lt;a href=&quot;https://gcc.gnu.org/wiki/avr-gcc&quot;&gt;Atmel AVR microcontrollers&lt;/a&gt; which is an important part of the Arduino IDE. Also, you’ll find GCC versions for &lt;a href=&quot;https://launchpad.net/gcc-arm-embedded&quot;&gt;ARM&lt;/a&gt; which e.g. the &lt;a href=&quot;https://forum.pjrc.com/threads/1600-Compile-GCC-for-teensy-3&quot;&gt;Teensy 3&lt;/a&gt; requires.&lt;/p&gt;

&lt;p&gt;Let’s shortly review some concepts behind the compilation tools. There are a number of commands to remember:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;em&gt;gcc/g++&lt;/em&gt;: This is the compiler. Compilers parse an input file from a human, and translate it to a binary file that a microprocessor understands. However, in our source code we likely want to reference code from other files and libraries. This is where a compiler sometimes acts a frontend to the linker.&lt;/li&gt;
  &lt;li&gt;&lt;em&gt;ld&lt;/em&gt;: You need a linker to bundle multiple binary files into one single file that runs on the microprocessor. Besides code organization, a linker reduces build time. You just re-compile files that changed and link everything into one file. You’ll find options on using g++ as linker &lt;a href=&quot;http://www.nongnu.org/avr-libc/user-manual/using_tools.html&quot;&gt;in the docs&lt;/a&gt;.&lt;/li&gt;
  &lt;li&gt;&lt;em&gt;ar&lt;/em&gt;:  Certain core libraries are linked straight into the target program. In order to make those libraries “linkable”, there is the &lt;a href=&quot;http://www.network-theory.co.uk/docs/gccintro/gccintro_79.html&quot;&gt;ar archiver&lt;/a&gt; to generate static libraries.&lt;/li&gt;
  &lt;li&gt;&lt;em&gt;objcopy&lt;/em&gt;: Binary files can still be in different formats. A popular format is &lt;a href=&quot;http://en.wikipedia.org/wiki/Executable_and_Linkable_Format&quot;&gt;ELF&lt;/a&gt; or “Executable and Linkable Format”. But some microprocessors require a different format. For example to upload to an Arduino, you need the “intel hex format”.  Converting between formats can be done with &lt;a href=&quot;http://ccrma.stanford.edu/planetccrma/man/man1/avr-objcopy.1.html&quot;&gt;objcopy&lt;/a&gt;.&lt;/li&gt;
  &lt;li&gt;&lt;em&gt;avrdude&lt;/em&gt;: Once your binary file is ready and in the correct format you need to upload it to the embedded device. This is what &lt;a href=&quot;http://www.nongnu.org/avrdude/&quot;&gt;avrdude&lt;/a&gt; does for Arduino.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A picture how these tools form a chain is shown &lt;a href=&quot;http://www.jusquici.org/blog/?page_id=202&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;The Arduino IDE will install all the the AVR GCC tools for you. You can also &lt;a href=&quot;http://nothingtodisplay.org/avr-toolchain-with-homebrew-mac-os-x/&quot;&gt;manually install the AVR GCC toolchain&lt;/a&gt; on a Mac:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ brew install avr-libc
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;On Linux, this should do the trick:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ sudo apt-get install gcc-avr binutils-avr gdb-avr avr-libc avrdude
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Windows users should find instructions &lt;a href=&quot;winavr.sourceforge.net&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;To follow the rest of this post, you can also use the AVR GCC tools that come with the Arduino IDE.&lt;/p&gt;

&lt;h1 id=&quot;compile-a-blink-sketch&quot;&gt;Compile a Blink sketch&lt;/h1&gt;

&lt;p&gt;Learning the secrets of compilers and build tools takes some practice. Let’s walk through a simplified build cycle to get you more familiar with the GCC tools.&lt;/p&gt;

&lt;p&gt;Imagine this &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;blink.ino&lt;/code&gt; Arduino sketch:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;void setup() {
	// initialize Pin13 as an output
	pinMode(13, OUTPUT);
}

void loop() {
	digitalWrite(13, LOW);   
	delay(1000);            
	digitalWrite(13, HIGH); 
	delay(400);            
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;To compile this sketch with &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;g++&lt;/code&gt;, you must include the &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;Arduino.h&lt;/code&gt; header and include a &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;main()&lt;/code&gt; function. This can be done simply by adding:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ echo &apos;#include &quot;Arduino.h&quot;&apos;  &amp;gt; blink.cpp
$ cat /Applications/Arduino.app/Contents/Java/hardware/arduino/avr/cores/arduino/main.cpp &amp;gt;&amp;gt; blink.cpp
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;With the &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;arduino.h&lt;/code&gt; header, you get functions to work with pins and ports. Also, you get function prototypes for the main parts of your sketch:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;void setup(void);
void loop(void);
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;blink.cpp&lt;/code&gt; is valid for compilation.&lt;/p&gt;

&lt;p&gt;You can compile this blink.cpp to a binary file with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ avr-g++ \
   -I /Applications/Arduino.app/Contents/Java/hardware/arduino/avr/cores/arduino/ -I /Applications/Arduino.app/Contents/Java//hardware/arduino/avr/variants/standard \
   -x c++  -MMD -c -mmcu=atmega328p -Wall -DF_CPU=16000000L  -Wall  -Os  blink.cpp
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;By using the -I compiler flag, we include the function prototypes of Arduino core libraries as we used &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;pinMode&lt;/code&gt; and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;digitalWrite&lt;/code&gt;. What follows are some harder to understand compiler flags that address the microprocessor type and its speed. We’ll see how we can get these automatically with a &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;Makefile? at the end of this post. Important right now: You get a binary file &lt;/code&gt;blink.o` after compilation.&lt;/p&gt;

&lt;h1 id=&quot;linking-libraries&quot;&gt;Linking Libraries&lt;/h1&gt;

&lt;p&gt;As we used the &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;pinMode&lt;/code&gt; and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;digitalWrite&lt;/code&gt; &lt;a href=&quot;https://github.com/arduino/Arduino/blob/master/hardware/arduino/avr/cores/arduino/wiring_digital.c&quot;&gt;functions&lt;/a&gt; of Arduino core libraries, we need to package some of these libraries into our &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;blink.o&lt;/code&gt; binary. This can be done with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ avr-ar rcs libcore.a hooks.o wiring.o wiring_digital.o
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The objects files from Arduino core libraries are obtained from compiling the corresponding C files (hooks.c, wiring.c and wiring_digital.c) from the Arduino folder.  By the way, the &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;hooks.o&lt;/code&gt; binary provides an empty helper to build Arduino projects without plugins.&lt;/p&gt;

&lt;p&gt;Then we link this together with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ avr-gcc -mmcu=atmega328p -Wl,--gc-sections -Os -o blink.elf blink.o libcore.a  -lc -lm
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;As last step, you must convert the elf binaries into the hex format:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ avr-objcopy -O ihex -R .eeprom blink.elf blink.hex
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The resulting hex file can then be uploaded to an Arduino UNO. An overview on the compilation steps can be found in this &lt;a href=&quot;https://gist.github.com/mulderp/36ca39a9911d54de66ec&quot;&gt;gist&lt;/a&gt;.&lt;/p&gt;

&lt;h1 id=&quot;makefile&quot;&gt;Makefile&lt;/h1&gt;

&lt;p&gt;Repeating these steps manually for every project would quickly become furstrating. To automate builds and &lt;a href=&quot;http://en.wikipedia.org/wiki/Configure_script&quot;&gt;configurations&lt;/a&gt;, there are Makefiles.&lt;/p&gt;

&lt;p&gt;A great start to build an Arduino Sketch with a Makefile is in the &lt;a href=&quot;https://github.com/sudar/Arduino-Makefile&quot;&gt;Arduino-Makefile&lt;/a&gt; project by &lt;a href=&quot;https://twitter.com/sudarmuthu&quot;&gt;@sudarmuthu&lt;/a&gt;. On my MacOS, I reference this project from my local project as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# Arduino Make file. Refer to https://github.com/sudar/Arduino-Makefile

BOARD_TAG    = uno

ARDUINO_DIR   = /Applications/Arduino.app/Contents/Java/
ARDMK_DIR     = ../../
AVR_TOOLS_DIR = /Applications/Arduino.app/Contents/Java/hardware/tools/avr/
MONITOR_PORT  = /dev/cu.usbmodem14131
BOARD_TAG     = uno
AVRDUDE_CONF = /Applications/Arduino.app/Contents/Java/hardware/tools/avr/etc/avrdude.conf

include ../../Arduino.mk
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h1 id=&quot;references&quot;&gt;References&lt;/h1&gt;

&lt;ul&gt;
  &lt;li&gt;Book: &lt;a href=&quot;http://www.network-theory.co.uk/gcc/intro/&quot;&gt;An introduction to GCC&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;A blog post that explains the different steps for compiling an Arduino sketch with &lt;a href=&quot;https://www.ashleymills.com/node/327&quot;&gt;GCC and Makefiles&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;A leaner &lt;a href=&quot;https://gist.github.com/wolever/273821&quot;&gt;Makefile&lt;/a&gt; for Arduino&lt;/li&gt;
  &lt;li&gt;Building an &lt;a href=&quot;http://angryelectron.com/stk500-arduino-bootloader/&quot;&gt;Arduino bootloader&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;Getting started with &lt;a href=&quot;http://www.atmel.com/images/doc1497.pdf&quot;&gt;AVR LIB-C&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;Some basics about &lt;a href=&quot;http://hackaday.com/2010/10/25/avr-programming-02-the-hardware/&quot;&gt;programming an Atmel ATmega&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;A basic &lt;a href=&quot;https://github.com/muccc/arm-workshop/blob/master/mvp/Makefile&quot;&gt;Makefile&lt;/a&gt; for ARM based MCU’s&lt;/li&gt;
  &lt;li&gt;Another discussions on &lt;a href=&quot;http://alejandroerickson.com/joomla/personal-projects/73-a-makefile-for-arduino-v-0021&quot;&gt;Makefiles&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</content>
 </entry>
 
 <entry>
   <title>Streaming data to the serial port</title>
   <link href="https://thinkingonthinking.com/serial-input-with-streams/"/>
   <updated>2015-03-16T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/serial-input-with-streams</id>
   <content type="html">&lt;p&gt;Many embedded systems have &lt;em&gt;no&lt;/em&gt; screens (as in the case of an Arduino) or only small screens (a wrist-watch for example). You already saw in &lt;a href=&quot;http://thinkingonthinking.com/serial-communication-with-nodejs/&quot;&gt;the last post&lt;/a&gt;, how you can read data from screen-less devices by listening on an UART with JavaScript.&lt;/p&gt;

&lt;p&gt;Now, let’s look at the opposite case:  &lt;em&gt;Writing&lt;/em&gt; bytes into a device. This becomes interesting for device configuration, feeding displays, adjusting light or parameters of a robot.&lt;/p&gt;

&lt;p&gt;As in the previous post, we use the serial port with JavaScript, and transmit data to an Arduino.  Before bytes can be transmitted, we must capture them, for example from a user.&lt;/p&gt;

&lt;p&gt;Here is an example for writing bytes over UART. Imagine you have a table football in your office. And, you want others allow to follow score on a 7 segment display.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://www.flickr.com/photos/merlijnhoek/2672429188&quot; title=&quot;Table football @ Fusion Festival 2008 (Sunday) by Merlijn Hoek, on Flickr&quot;&gt;&lt;img src=&quot;https://farm4.staticflickr.com/3090/2672429188_48356e4495.jpg&quot; width=&quot;500&quot; height=&quot;334&quot; alt=&quot;Table football @ Fusion Festival 2008 (Sunday)&quot; /&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;So, for this project you need to setup several things:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Capture input from a user with JavaScript and write to an UART&lt;/li&gt;
  &lt;li&gt;Display the score on a 7-segment display with an Arduino&lt;/li&gt;
&lt;/ul&gt;

&lt;h1 id=&quot;nodejs-streams&quot;&gt;NodeJS streams&lt;/h1&gt;

&lt;p&gt;A fundamental concept behind moving bytes in NodeJS are &lt;a href=&quot;https://nodejs.org/api/all.html#all_stream&quot;&gt;streams&lt;/a&gt;. With streams, you can observe and manipulate the flow of data over time. A good examples for streams are HTTP requests and responses. See &lt;a href=&quot;https://nodejs.org/api/http.html#http_class_http_clientrequest&quot;&gt;the NodeJS HTTP API docs for details&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Besides data in networks or databases, streams are nice to deal with user input too. Input from a user can be captured with a simple &lt;em&gt;writable&lt;/em&gt; stream:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var stream = require(&apos;stream&apos;);
var Stream = stream.Stream;

var ws = new Stream;
ws.writable = true;

ws.write = function(data) {
  console.log(&quot;input=&quot; + data);
}

ws.end = function(data) {
  console.log(&quot;bye&quot;);
}

process.stdin.pipe(ws);
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;A simple test shows how this works:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ node pipe_out.js
hello
input=hello
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The writeable stream handles “write” and “end” events from the standard input. This connection is made by &lt;em&gt;piping&lt;/em&gt; standard input to the writable stream. You could also pipe the output of a file into the write stream, e.g. with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ echo hello | node pipe_out.js
input=hello

bye
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;So far about the basics of a writeable stream. &lt;a href=&quot;http://twitter.com/substack&quot;&gt;@substack&lt;/a&gt; has &lt;a href=&quot;https://www.youtube.com/watch?v=lQAV3bPOYHo&quot;&gt;a great talk&lt;/a&gt; about details and types of streams.&lt;/p&gt;

&lt;h1 id=&quot;transmitting-data&quot;&gt;Transmitting data&lt;/h1&gt;

&lt;p&gt;As previously with the read command, you can simply &lt;em&gt;write&lt;/em&gt; to the serial port on write events. When you include the serial port in the previous example, the code becomes:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var Stream = require(&apos;stream&apos;);
var modem = &apos;cu.usbmodem14231&apos;;

var ws = new Stream;
ws.writable = true;

ws.write = function(data) {
  serialPort.write(data);
};

ws.end = function(buf) {
  console.log(&apos;bye&apos;);
}

var serialPort = new SerialPort(&apos;/dev/&apos; + modem, {
  baudrate: 9600,
  parser: serialport.parsers.readline(&quot;\n&quot;)
});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now, you could connect to some periphery and send data with 9600 baud. Let’s explore that. First we must again prepare the Arduino.&lt;/p&gt;

&lt;h1 id=&quot;the-arduino-side&quot;&gt;The Arduino side&lt;/h1&gt;

&lt;p&gt;On the Arduino, we now need to examine its UART whether there is data in the receive buffer. If so, we parse the data and the do something, e.g. activating a display.&lt;/p&gt;

&lt;p&gt;What follows is some simple Arduino code to read two numbers separated by a character:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;  if (Serial.available() &amp;gt; 0) { 
    incoming[0] = Serial.parseInt(); 
    incoming[1] = Serial.parseInt(); 

    if (Serial.read() == &apos;\n&apos;) {
       // light the display
    }
  }
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;More info can be found this can be found in the &lt;a href=&quot;http://arduino.cc/en/Tutorial/ReadASCIIString&quot;&gt;ReadASCIIString example&lt;/a&gt; and the &lt;a href=&quot;http://arduino.cc/en/Reference/ParseInt&quot;&gt;parseInt API docs&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;An nice 7 segment display to see some action is &lt;a href=&quot;https://www.sparkfun.com/products/11442&quot;&gt;this product from Sparkfun&lt;/a&gt;.
&lt;a href=&quot;https://github.com/sparkfun/Serial7SegmentDisplay/blob/master/firmware/Serial%207-Segment%20Display/Arduino_Examples/S7S_Example_Serial_ColonDots/S7S_Example_Serial_ColonDots.ino&quot;&gt;Some examples&lt;/a&gt; illustrate how to control the numbers on the screen.&lt;/p&gt;

&lt;h1 id=&quot;the-result&quot;&gt;The result&lt;/h1&gt;

&lt;p&gt;The result? Now, you can write the scores of a table football match or any other game to an external display.  We are going to learn soon how to attach web services and interfaces to hardware.  Then streams and NodeJS can start to show its real benefits.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://www.flickr.com/photos/pmulder99/16238050843&quot; title=&quot;display by Patrick Mulder, on Flickr&quot;&gt;&lt;img src=&quot;https://farm9.staticflickr.com/8563/16238050843_db297cb3d3.jpg&quot; width=&quot;375&quot; height=&quot;500&quot; alt=&quot;display&quot; /&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Note: This post is the first post of a new series about JavaScript for embedded devices. This will be the topic of my new book. Any feedback, suggestions and criticism is highly appreciated.&lt;/em&gt;&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Serial communication with NodeJS</title>
   <link href="https://thinkingonthinking.com/serial-communication-with-nodejs/"/>
   <updated>2015-03-14T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/serial-communication-with-nodejs</id>
   <content type="html">&lt;p&gt;How can you talk to hardware peripherals, lights or robots from your laptop?&lt;/p&gt;

&lt;p&gt;For many cases, the answer is: &lt;a href=&quot;http://en.wikipedia.org/wiki/Serial_communication&quot;&gt;Serial communication&lt;/a&gt;.  With serial communication, you can send bits and bytes “in series” over a wire.  This often looks as in the figure below.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/serial_comm.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;While you will always find a wire for data, the clock is optional in some communication schemes. The clock can be included for accurate timing of data signals. But in other communication schemes, the clock is left out. From operating system perspective, a nice overview is &lt;a href=&quot;https://www.freebsd.org/doc/en/articles/serial-uart/&quot;&gt;here&lt;/a&gt;. This is possible when timing is defined on the sending and receiving ends, for example when data transmission should happen on 57600 bits per second.&lt;/p&gt;

&lt;p&gt;Besides timing, serial communication is about configuration of wires. Data travels in one or multiple directions, usually on wires. But serial communication concepts often apply to wire-&lt;em&gt;less&lt;/em&gt; communication too.&lt;/p&gt;

&lt;p&gt;There are different forms of serial communication such as &lt;a href=&quot;http://en.wikipedia.org/wiki/RS-232&quot;&gt;RS-232&lt;/a&gt; or &lt;a href=&quot;http://en.wikipedia.org/wiki/USB&quot;&gt;USB&lt;/a&gt;. In both cases, the core building block in a micro chip is an &lt;a href=&quot;http://en.wikipedia.org/wiki/Universal_asynchronous_receiver/transmitter&quot;&gt;UART&lt;/a&gt; (or “universal asynchronous receiver and transmitter”).  This digital logic can be configured to different data formats and transmission speeds.&lt;/p&gt;

&lt;p&gt;On your computer, an operating system connects to the UART with &lt;em&gt;a driver&lt;/em&gt;. The driver takes care of inputs and outputs, and configuration of the signals (or &lt;a href=&quot;http://en.wikipedia.org/wiki/Ioctl&quot;&gt;ioctl&lt;/a&gt; in computing language). This means, if you want to talk to your Arduino from your laptop with NodeJS, your JavaScript actually talks to the driver for the serial port.&lt;/p&gt;

&lt;p&gt;Sounds complicated? Maybe some code examples clear things up.&lt;/p&gt;

&lt;h1 id=&quot;the-serialport-library&quot;&gt;The serialport library&lt;/h1&gt;

&lt;p&gt;JavaScript and hardware - why would you combine both? There are some good arguments about &lt;a href=&quot;http://www.voodootikigod.com/nodebots-the-rise-of-js-robotics/&quot;&gt;using JavaScript with hardware&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;The physical world is accessible for JavaScript projects thanks to the &lt;a href=&quot;https://www.npmjs.com/package/serialport&quot;&gt;serialport module&lt;/a&gt; by &lt;a href=&quot;https://github.com/voodootikigod&quot;&gt;Chris Williams&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Let’s create a fresh JavaScript project and add this module to it:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ npm install --save-dev serialport
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;After the module has been downloaded, you are ready to start some serial communication fun.&lt;/p&gt;

&lt;h1 id=&quot;opening-a-port&quot;&gt;Opening a port&lt;/h1&gt;

&lt;p&gt;Before any communication can happen, you must &lt;em&gt;open&lt;/em&gt; the serial port. This is the start of every program. In the file “dump_usb.js”, you write:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var serialport = require(&quot;serialport&quot;); 
var SerialPort = serialport.SerialPort; 

var serialPort = new SerialPort(&quot;/dev/cu.usbmodem14131&quot;, {
  baudrate: 9600,
  parser: serialport.parsers.readline(&quot;\n&quot;)
});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;You must pass the port name, something like “COM1” on Windows or “/dev/ttyS0” on POSIX platforms. In case of my Arduino, it is “/dev/cu.usbmodem14131”.&lt;/p&gt;

&lt;p&gt;Next, you must configure the port, mainly the transmission speed for sending and receiving bits.  (Serial port configuration can also include things as flow control type, parity, stop bits and character size - this is out of scope for most Arduino projects).&lt;/p&gt;

&lt;p&gt;For Arduino projects, a good choice for transmission speed is 9600 baud, because you can still watch incoming bits “live”. If there is a speed mismatch between your computer and your Arduino, you’ll see weird output.&lt;/p&gt;

&lt;p&gt;As last argument for opening the port, you better set a &lt;em&gt;parser&lt;/em&gt; for the incoming data. The default readline parser is nice to get line breaks out of the way.&lt;/p&gt;

&lt;h1 id=&quot;prepare-the-arduino&quot;&gt;Prepare the Arduino&lt;/h1&gt;

&lt;p&gt;Once the serial port is open, we have access to its data stream. The easiest thing to do is dumping data from the stream to the terminal.&lt;/p&gt;

&lt;p&gt;Let’s have the Arduino send some data over with the &lt;a href=&quot;http://arduino.cc/en/reference/serial&quot;&gt;Serial library&lt;/a&gt;. The following Arduino sketch “counter.ino” increases a counter, and sends the value over:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;setup() {
  Serial.begin(9600);
}

int i=0;
void loop() {
  Serial.println(i++);
  delay(100); // poll every 100ms
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The last “delay” gives the laptop some time to process the data. Providing a delay is a good idea when you want to watch a data stream with human eyes instead of a CPU’s.&lt;/p&gt;

&lt;h1 id=&quot;reading-from-the-port&quot;&gt;Reading from the port&lt;/h1&gt;

&lt;p&gt;To read bits from the port, you must add some more lines to the “dump_usb.js” script:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var serialport = require(&quot;serialport&quot;);
var SerialPort = serialport.SerialPort;

var serialPort = new SerialPort(&quot;/dev/cu.usbmodem14131&quot;, {
  baudrate: 9600,
  parser: serialport.parsers.readline(&quot;\n&quot;)
});

serialPort.on(&quot;open&quot;, function () {
  console.log(&apos;open&apos;);
  serialPort.on(&apos;data&apos;, function(data) {
    console.log(data);
  });
});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The last part is important: We listen to the &lt;em&gt;open&lt;/em&gt; event from the driver. Then, we subscribe to its &lt;em&gt;data&lt;/em&gt; events. The received bits are displayed to the screen with &lt;em&gt;console.log(data)&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;Let’s run this script:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ node dump_usb.js
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;If everything works out, you’ll see some numbers from the Arduino scrolling down on the screen. With a bit more advanced Arduino breadboard setup (see the screenshot), you can easily measure the room temperature or the position of a potentiometer.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://www.flickr.com/photos/pmulder99/16623712038&quot; title=&quot;arduino serial by Patrick Mulder, on Flickr&quot;&gt;&lt;img src=&quot;https://farm9.staticflickr.com/8716/16623712038_89579b2ec8_z.jpg&quot; width=&quot;480&quot; height=&quot;640&quot; alt=&quot;arduino serial&quot; /&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h1 id=&quot;conclusion&quot;&gt;Conclusion&lt;/h1&gt;

&lt;p&gt;This post gave you some basic concepts for communication with an embedded device with NodeJS. Not too much happened at this stage. Just numbers scrolling down the screen. Things will get more fun in a future blog post, where we’ll connect the serial port to the browser by using a simple “polling” mechansism. This will enable us to write some simple web interfaces for hardware.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Note: This post is the first post of a new series about JavaScript for embedded devices. This will be the topic of my new book. You can track its progress on the &lt;a href=&quot;http://embeddednodejs.com/chapters.html&quot;&gt;book website&lt;/a&gt;. Any feedback, suggestions and criticism is highly appreciated.&lt;/em&gt;&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Digital and analog interactions</title>
   <link href="https://thinkingonthinking.com/digital-and-analog/"/>
   <updated>2015-03-01T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/digital-and-analog</id>
   <content type="html">&lt;p&gt;It was great to learn about a mash up of interfaces, hardware and software that is generally labelled “Internet of Things”. &lt;a href=&quot;https://www.youtube.com/watch?v=QB0ordd2nOI&quot;&gt;Massimo Banzi&lt;/a&gt; from &lt;a href=&quot;http://arduino.cc/&quot;&gt;Arduino&lt;/a&gt; provided a lot of interesting material for doing further experiments.&lt;/p&gt;

&lt;h1 id=&quot;designing-smart-interactions&quot;&gt;Designing smart interactions&lt;/h1&gt;

&lt;p&gt;Sitting in one of IDEO Munich’s creative spaces, the workshop started with general design discussions about needs for connected devices. When you start listing devices of daily usage, you quickly discover many sensors and feedback loops to run our everyday lifes. In the next years, Internet of Things offers smarter devices by recording, communicating and visualizing patterns about interactions with these devices.&lt;/p&gt;

&lt;p&gt;After brainstomring connected devices, we learnt how the &lt;a href=&quot;http://arduino.cc/en/Guide/ArduinoYun&quot;&gt;Arduino Yun&lt;/a&gt; can help a lot in prototyping interactions for the Internet of Things. The Arduino Yun combines Wifi and ethernet capabilities with an &lt;a href=&quot;http://arduino.cc/en/Main/arduinoBoardLeonardo&quot;&gt;Arduino Leonardo&lt;/a&gt; which has among other things 6 analog &lt;em&gt;inputs&lt;/em&gt; and 6 analog &lt;em&gt;outputs&lt;/em&gt;.&lt;/p&gt;

&lt;h2 id=&quot;sidenote-the-ivrea-design-school&quot;&gt;Sidenote: The Ivrea design school&lt;/h2&gt;
&lt;p&gt;Arduino and its role for interaction design were rooted in the &lt;a href=&quot;http://interactionivrea.org/en/index.asp&quot;&gt;Ivrea design school&lt;/a&gt;. Italy is famous for design, and Ivrea in particular for building Olivetti. Olivetti experimented with a design culture beyond their products as this video shows.&lt;/p&gt;

&lt;iframe width=&quot;420&quot; height=&quot;315&quot; src=&quot;https://www.youtube.com/embed/97O5Iyb8Sz0&quot; frameborder=&quot;0&quot; allowfullscreen=&quot;&quot;&gt;&lt;/iframe&gt;

&lt;p&gt;The Ivrea design school collaborated with many important designers and design thinkers, such as &lt;a href=&quot;http://www.designinginteractions.com/bill&quot;&gt;Bill Moggridge&lt;/a&gt;, co-founder of IDEO. To grasp an idea of his work, this video is interesting.&lt;/p&gt;

&lt;iframe width=&quot;420&quot; height=&quot;315&quot; src=&quot;https://www.youtube.com/embed/kVkQYvN4_HA&quot; frameborder=&quot;0&quot; allowfullscreen=&quot;&quot;&gt;&lt;/iframe&gt;

&lt;h1 id=&quot;no-screens&quot;&gt;No screens&lt;/h1&gt;

&lt;p&gt;Back to the workshop, when building devices with Arduino, it quickly becomes clear: There are no screens. You can read and write things via the Serial monitor. But a lot of physical computing is done directly with light, motion or sound. This is rather different when you are used to build UIs with software only.&lt;/p&gt;

&lt;p&gt;Still, when gathering data from sensors, visualizations play an important role. We used &lt;a href=&quot;http://wiki.sandaysoft.com/a/SteelSeries_Gauges&quot;&gt;this simple gauge library&lt;/a&gt; to show data from remote sensors.&lt;/p&gt;

&lt;p&gt;Before you can visualize data, you must fetch it over the network. Here, we saw the role of &lt;a href=&quot;http://mqtt.org/&quot;&gt;MQTT&lt;/a&gt;, &lt;a href=&quot;http://hivemq.com/demos/websocket-client/&quot;&gt;HiveMQ&lt;/a&gt; and &lt;a href=&quot;http://test.mosquitto.org/&quot;&gt;Mosquitto&lt;/a&gt;. Massimo provided many details about the background of this protocol, and why the publish-subscribe pattern is so important for Internet of Things.&lt;/p&gt;

&lt;h1 id=&quot;apis&quot;&gt;APIs&lt;/h1&gt;

&lt;p&gt;Services for connecting devices were the last topic of the workshop. By skipping the software UI’s, this part led to questions about how to wrap APIs. Working with &lt;a href=&quot;https://www.temboo.com/library/Library/NYTimes/TimesNewswire/GetRecentNewsItems/&quot;&gt;Temboo&lt;/a&gt;, we quickly had some working examples of reading and writing messages to Twitter, for example.&lt;/p&gt;

&lt;h1 id=&quot;conclusions&quot;&gt;Conclusions&lt;/h1&gt;

&lt;p&gt;It was a great weekend. By putting many simple ideas in a new context, it was interesting to get a feeling for the future: Connected worlds. Connectivity, however, is still not easy these days. What feels seamless transmission over the air, are often highly engineerd software solutions around Wifi protocols. Yet, combining interfaces, circuits and code is very nice to explore.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>JSONP example with BackboneJS</title>
   <link href="https://thinkingonthinking.com/jsonp-with-backbonejs/"/>
   <updated>2015-01-15T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/jsonp-with-backbonejs</id>
   <content type="html">&lt;p&gt;The idea of &lt;a href=&quot;http://en.wikipedia.org/wiki/JSONP&quot;&gt;JSONP API’s&lt;/a&gt; is nice: Many Ajax applications can’t fetch JavaScript objects from a remote domain due to &lt;a href=&quot;http://www.staticapps.org/articles/cross-domain-requests-with-cors&quot;&gt;cross-domain policy concerns&lt;/a&gt;. But, JavaScript web apps can easily fetch remote JavaScript libraries. So, by using JSONP, we pretend fetching a library, while we actually are interested in the data.&lt;/p&gt;

&lt;p&gt;Unfortunately, JSONP has some &lt;a href=&quot;http://security.stackexchange.com/questions/23438/security-risks-with-jsonp&quot;&gt;security flaw in the context of CSRF (cross-site request forgery)&lt;/a&gt;.  A 3rd user could make HTTP requests on your behalf, because a server would trust your cookie. I am not a security expert, but many JSONP API providers disable writing.&lt;/p&gt;

&lt;p&gt;For simple projects, JSONP can make sense however - for example, if you would like to provide a nicer UI to someone else’s &lt;a href=&quot;https://wordpress.org/support/topic/getting-jsonp-to-work&quot;&gt;data in Wordpress&lt;/a&gt;. This was a question of some reader of &lt;a href=&quot;http://pipefishbook.com&quot;&gt;The Pipefishbook&lt;/a&gt;. So, let’s look at a Backbone.Collection that reads data with JSONP.&lt;/p&gt;

&lt;h1 id=&quot;ajax-requests&quot;&gt;Ajax requests&lt;/h1&gt;

&lt;p&gt;Fetching data with an Ajax request is easy:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$.get(&apos;/some/path&apos;);
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;&lt;a href=&quot;http://api.jquery.com/jquery.ajax/&quot;&gt;jQuery Ajax requests&lt;/a&gt; help you very much to wrap a JSONP request for data too. For the details, you can search &lt;a href=&quot;http://npmjs.org&quot;&gt;npmjs.org&lt;/a&gt; for JSONP. You’ll find a &lt;a href=&quot;https://github.com/webmodules/jsonp/blob/master/index.js#L87&quot;&gt;JSONP module&lt;/a&gt; and the according tricks. Append a  &amp;lt;script&amp;gt; tag, and point the source towards your remote data.&lt;/p&gt;

&lt;p&gt;A Backbone.Collection can deal with those requests too. Let’s look an example based on the Flickr API.&lt;/p&gt;

&lt;h1 id=&quot;browsing-images-with-jsonp&quot;&gt;Browsing images with JSONP&lt;/h1&gt;

&lt;p&gt;Let’s look at a small example to &lt;a href=&quot;http://flickr-example.divshot.io/&quot;&gt;browse photos from Flickr&lt;/a&gt;. You can see the full &lt;a href=&quot;https://github.com/pipefishbook/jsonp-example&quot;&gt;source code at Github&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;This demo is based on a parameterized Backbone.Collection.  Wrapping Ajax requests is an important responsibility of the Backbone data layer. So, let’s look at &lt;a href=&quot;https://github.com/pipefishbook/jsonp-example/blob/master/app/collections/photos.js&quot;&gt;a collection to fetch photos with JSONP&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;It is important to add a “dataType” parameter to the Ajax request. This parameter is then passed to te jQuery Ajax call:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; sync: function(method, collection, options) {
   options.dataType = &quot;jsonp&quot;;
   options.jsonpCallback = &quot;jsonpCallback&quot;;
   return Backbone.sync(method, collection, options);
 }
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And this is already all. The rest of the the example uses much of the same view construction and controls as in &lt;a href=&quot;http://pipefishbook.com/ch_4/subviews/&quot;&gt;Ch. 4 example&lt;/a&gt; of the Pipefishbook. A subviews for control and a list view share the same collection. Collection are fetched just upon request (= button click).&lt;/p&gt;

&lt;p&gt;A nice extra for this post: Sometimes, you have a JSON API that you can’t directly access due to cross-policy browser concerns. Fortunately, there is a &lt;a href=&quot;https://jsonp.nodejitsu.com/&quot;&gt;proxy from Nodejitsu&lt;/a&gt; to emulat e JSONP request. This allows you to easily fetch data from JSON API without setting up a proxy yourself.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Browserify, Makefiles and Vim</title>
   <link href="https://thinkingonthinking.com/makefiles-vim-and-browserify/"/>
   <updated>2015-01-07T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/makefiles-vim-and-browserify</id>
   <content type="html">&lt;p&gt;Build processes can be obscure. Especially, if you use IDE’s with heavy automation.&lt;/p&gt;

&lt;p&gt;As I currently work solo on JavaScript projects, which often are prototypes, I prefer to keep my build processes small. In fact, I like to use “bare metal” tools with maximum control. These tools are &lt;a href=&quot;http://www.vim.org/download.php&quot;&gt;Vim&lt;/a&gt;, &lt;a href=&quot;http://browserify.org/&quot;&gt;Browserify&lt;/a&gt;, SuperStatic and &lt;em&gt;Makefiles&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;So, read on what might happen &lt;a href=&quot;http://thinkingonthinking.com/fast-start-with-backbonejs/&quot;&gt;after your fast start with Backbone&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;PS This post is also a reflection on the critique on “too much automation” in &lt;a href=&quot;http://pipefishbook.com&quot;&gt;The Pipefishbook&lt;/a&gt;, where I discussed tools such as Yeoman or Grunt together with Backbone. These build tools can make sense - but this is another blog post. Short answer: If you work towards frameworks to address many users and platforms, an IDE strategy can help. But prototypes are different.&lt;/p&gt;

&lt;h1 id=&quot;vim&quot;&gt;VIM&lt;/h1&gt;

&lt;p&gt;VIM is an editor from the &lt;a href=&quot;http://www.binpress.com/blog/2014/11/19/vim-creator-bram-moolenaar-interview/&quot;&gt;early 90ies&lt;/a&gt;. It has a powerful script engine built-in. With this, you can easily add “mappings” to run certain commands to build some files, or, to start a web server. These commands avoid maintaining “expensive” live-reload processes, and get you almost the same amount of fast feedback.&lt;/p&gt;

&lt;p&gt;Some of my efficient key mappings for JavaScript projects are these in the file .vimrc.&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;let mapleader=&quot;,&quot;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This mapping defines some meta-key or “leader” such as “,”. The leader key is used to avoid interference with other VIM commands.&lt;/p&gt;

&lt;p&gt;With this leader key, you can define mappings to cycle through buffers quickly. For example, you can define a key mapping leader-leader as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;noremap &amp;lt;leader&amp;gt;&amp;lt;leader&amp;gt; :bp &amp;lt;cr&amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The real fun starts, when you trigger a browserify build from another mapping:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;noremap &amp;lt;leader&amp;gt;b :!browserify ./app/main.js &amp;gt; static/bundle.js&amp;lt;cr&amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And, to start a static web server on a command, I use:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;noremap &amp;lt;leader&amp;gt;ss :!ss static &amp;lt;cr&amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;With these basic commands, you can easily write, edit and build things. No advanced IDE or build processes required.&lt;/p&gt;

&lt;h1 id=&quot;browserify&quot;&gt;Browserify&lt;/h1&gt;

&lt;p&gt;Editors only get you so far in coding JavaScript. I like Browserify to package JavaScript modules for the browser. Browserify &lt;a href=&quot;http://thinkingonthinking.com/unix-in-the-browser/&quot;&gt;brings Unix to the browser&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;The main browserify options are:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;Using external JavaScript libs with ‘-x’: This is nice to maintain e.g. jQuery outside of your app bundle. I use this option to work with jQuery object in the browser console too, e.g. to try Ajax requests.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Bundling modules as &lt;em&gt;external&lt;/em&gt; to use “require(…)” from the browser console: This is a nice option to work with e.g. a Backbone.Collection from the browser console, as well as from your app.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Browserify transforms can be used to translate HTML templates or other assets into valid JavaScript code.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Once you browserified your project, and you reference a bundle.js from a HTML file, I like to serve these together with &lt;a href=&quot;https://www.npmjs.com/package/superstatic&quot;&gt;SuperStatic&lt;/a&gt;. And, as above, I like to start the server from VIM. This usually serves your app from localhost:3474.&lt;/p&gt;

&lt;h1 id=&quot;makefile&quot;&gt;Makefile&lt;/h1&gt;

&lt;p&gt;When your build process becomes a bit more complicated, typing long commands is boring and error-prone. This is where Makefiles come in. I usually have a some definitions (= targets) for different browserify options.&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;ext:    ## use jquery as external reference
	browserify -x jQuery ./app/main.js &amp;gt; static/bundle.js
  
jst:
	browserify -t jstify ./app/main.js &amp;gt; static/bundle.js
  
eco:
  	browserify -t browserify-eco ./app/main.js &amp;gt; static/bundle.js
  
watch:
  	watchify -t jstify ./app/main.js -o static/bundle.js
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now, depending on the stage of a prototype, I can easily switch strategies: Using an external JQuery, watchify assets, check templating engines, …
James Coglan has a written a &lt;a href=&quot;https://blog.jcoglan.com/2014/02/05/building-javascript-projects-with-make/&quot;&gt;great overview on the roles of Makefiles in JavaScript projects&lt;/a&gt;.&lt;/p&gt;

&lt;h1 id=&quot;conclusions&quot;&gt;Conclusions&lt;/h1&gt;

&lt;p&gt;You can experiment and prototype JavaScript apps very well without much automation and distractions. In fact, for my currents work, the tools above still provide great value. There are times when a &lt;a href=&quot;https://www.npmjs.com/package/generator-backbone-browserify&quot;&gt;Yeoaman generator&lt;/a&gt; makes sense. And I will write about it soon hopefully. There are also times when Gruntfiles make sense, since Makefiles were not very well supported on Windows.&lt;/p&gt;

&lt;p&gt;Thanks for reading and sharing feedback. Happy hacking.&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Re-entering the Hardware business</title>
   <link href="https://thinkingonthinking.com/re-entering-hardware/"/>
   <updated>2015-01-02T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/re-entering-hardware</id>
   <content type="html">&lt;p&gt;Happy new year everyone. In the spirit of sharing New Year’s resolutions, here is some loud thinking about what happened the last months, and where it might lead to.&lt;/p&gt;

&lt;p&gt;Several months have passed since my &lt;a href=&quot;http://thinkingonthinking.com/first-week-as-freelancer/&quot;&gt;first post about freelancing&lt;/a&gt;. In the meanwhile, I am still working and enjoying hardware development, also in 2015. Short reminder: &lt;a href=&quot;http://thinkingonthinking.com/switching-blogs/&quot;&gt;I found out&lt;/a&gt; that my best marketable skill in Munich was still hardware.&lt;/p&gt;

&lt;p&gt;While catching up with electronics and soldering, I believe that the web platform and hardware are a great fit. To explore this, I founded the &lt;a href=&quot;http://munich-aug.de&quot;&gt;Munich Arduino &amp;amp; Co meetup&lt;/a&gt; and joined the &lt;a href=&quot;http://munichmakes.de&quot;&gt;Munich MCSM Maker Space&lt;/a&gt;. So far, it was very interesting  to see that many members of the &lt;a href=&quot;http://munich-aug.de&quot;&gt;meetup&lt;/a&gt; are actually programmers. For sure, data from robots and our environments will provide many interesting problems for software developers too.&lt;/p&gt;

&lt;p&gt;While having less time for pogramming, I am still exploring prototypes and simple web projects too. I wrote about how &lt;a href=&quot;http://thinkingonthinking.com/javascript-matters-for-prototypes/&quot;&gt;JavaScript matters for prototypes&lt;/a&gt;. I still love how Node and JavaScript work from the command-line and in the browser. (I guess &lt;a href=&quot;http://thinkingonthinking.com/unix-in-the-browser/&quot;&gt;Browserify - Unix in the browser&lt;/a&gt; has been my most popular post last year).&lt;/p&gt;

&lt;p&gt;With less programming, I re-learned the importance of communication about problems. Instead of starting with code (or tests) in the first place, it was more important to get a feeling about the problem space. And by discussing problems with others, theirs and yours, you often discover new solutions, without touching a keyboard. And, it sometimes meant, that I had  to run macros in Excel…&lt;/p&gt;

&lt;p&gt;More highlights last year were certainly &lt;a href=&quot;http://make-munich.de/&quot;&gt;Make Munich&lt;/a&gt; and &lt;a href=&quot;http://www.electronica.de/&quot;&gt;Electronica&lt;/a&gt;, and &lt;a href=&quot;http://push-conference.com/&quot;&gt;Push Conference&lt;/a&gt; where I met Ben Fry. All events provided very interesting interactions and feedback on ideas to further combine hardware, software and visual languages. &lt;a href=&quot;http://vcard.pipefishbook.com/&quot;&gt;A very small experiment&lt;/a&gt; from last month is the VCard pixel editor, which uses Backbone.js to convert pixels into hex codes. I hope to show more interactive hardware in the browser the next months.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Dealing with finance - A book review</title>
   <link href="https://thinkingonthinking.com/book-review-python-for-finances/"/>
   <updated>2014-12-04T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/book-review-python-for-finances</id>
   <content type="html">&lt;p&gt;People who work in finance work with spreadsheets. But spreadsheets have some problems:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;Accessing spreadsheets from outside is hard. In data networks for example, it can be important to send a link or details of transactions to clients. In contrast to &lt;a href=&quot;https://www.ietf.org/rfc/rfc2616.txt&quot;&gt;HTTP&lt;/a&gt;, “sending spreadsheets” is much harder for data sharing and collaboration.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;For advanced visualizations, data in “raw” format, e.g. &lt;a href=&quot;http://en.wikipedia.org/wiki/Comma-separated_values&quot;&gt;CSV&lt;/a&gt; is much easier to visualize with libraries such as &lt;a href=&quot;http://d3js.org/&quot;&gt;D3.js&lt;/a&gt;.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;While you can easily manipulate data in cells of a spreadsheet, it is hard to build &lt;a href=&quot;http://thinkingonthinking.com/Bookshelf-an-ORM-for-Node/&quot;&gt;a language around your data&lt;/a&gt;. For example, if you want to apply different tax or interest rates according to some rules, how would you communicate these rules in a spreadsheet?&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;From working in &lt;a href=&quot;http://fidor.de&quot;&gt;a bank&lt;/a&gt; (until last year), I can tell you that processing numbers from spreadsheets and &lt;a href=&quot;http://en.wikipedia.org/wiki/Comma-separated_values&quot;&gt;CSV&lt;/a&gt; files is the daily business in finance.&lt;/p&gt;

&lt;p&gt;Somehow it is strange, that we discuss crypto-currencies in the web, but most financial data still hides in spreadsheets.&lt;/p&gt;

&lt;p&gt;So, I had some hidden hopes, when I looked at the book &lt;a href=&quot;http://shop.oreilly.com/product/0636920032441.do&quot;&gt;Python for Finance&lt;/a&gt;. While the book is not about open-source finance (for running a business or just to manage your personal cashflow), it has some interesting ingredients.&lt;/p&gt;

&lt;p&gt;Keeping track of finances is difficult enough. Most data comes as time-series data, but financial decisions often relate to risks as well. In the book, the first chapters will get you a basic taste of financial models and formula’s for understanding trends in financial markets.&lt;/p&gt;

&lt;p&gt;These chapters feel a bit academic. For an outsider, it is hard to understand why you want to try the effects of different parameters in the &lt;a href=&quot;http://en.wikipedia.org/wiki/Black%E2%80%93Scholes_model&quot;&gt;Black-Scholes formula&lt;/a&gt;. From what I can tell, the book example does a bit the same as the example on &lt;a href=&quot;http://www.wolframalpha.com/input/?i=black+scholes&amp;amp;lk=4&quot;&gt;WolframAlpha&lt;/a&gt;. But, how or where does my financial data fit into this game?&lt;/p&gt;

&lt;p&gt;If I would write a finance book for programmers, I would start with CSV. The book would show the advantages of CSV: Its openess, ease of use, hackability. Also, CSV can be used for small and big data. This year, there was even &lt;a href=&quot;http://csvconf.com/&quot;&gt;a CSV conference&lt;/a&gt; in Berlin. Maybe some libraries could be mentioned that allow access to finance APIs.&lt;/p&gt;

&lt;p&gt;Before “Python for Finance” mentions CSV, it continues with Python libraries for analyzing numbers, e.g. with the &lt;a href=&quot;http://pandas.pydata.org/&quot;&gt;Pandas library&lt;/a&gt;. Some early words on getting up and running with Python would be nice. Especially, how to use the Python package manager, &lt;em&gt;before&lt;/em&gt; any Python code is introduced.&lt;/p&gt;

&lt;p&gt;After the Black-Scholes examples, the books introduces &lt;a href=&quot;https://store.continuum.io/cshop/anaconda/&quot;&gt;Anaconda&lt;/a&gt; and &lt;a href=&quot;http://ipython.org/&quot;&gt;IPython&lt;/a&gt;. On a quick glance, I could not see, what Python flavor should be used when. and how these package are different from my “native” system Python on MacOS.&lt;/p&gt;

&lt;p&gt;What follows in the book are impressive plots based on financial models and simulations. I have never seen these kinds of plots outside of Matlab worlds. Exploring the books’ examples for visualizations would certainly be interesting, if you feel passionate about the stock market. For my taste, it would be interesting to understand time-series financial data in other contexts too, for example for corporate finance purposes, or just for estimating my income taxes.&lt;/p&gt;

&lt;p&gt;Well, all in all, while working with Matlab currently in electronic engineering, I can see the value of scripting simulations or plots with Python. The book gives a nice introduction to it, but it certainly won’t help you understanding the finances of a business or your personal income and expenses.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Disclaimer&lt;/em&gt;: This post was written in the context of the &lt;a href=&quot;http://oreilly.com/reviews&quot;&gt;O’Reilly Reader Review program&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;http://www.oreilly.com/reviews/?cmp=ex-orm-blgr-patrick-mulder&quot;&gt;&lt;img alt=&quot;I review for the O&apos;Reilly Reader Review Program&quot; src=&quot;http://cdn.oreillystatic.com/bloggers/blogger-review-badge-200.png&quot; border=&quot;0&quot; width=&quot;200&quot; height=&quot;150&quot; /&gt;&lt;/a&gt;&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>A faster start with Backbone.js</title>
   <link href="https://thinkingonthinking.com/fast-start-with-backbonejs/"/>
   <updated>2014-11-27T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/fast-start-with-backbonejs</id>
   <content type="html">&lt;p&gt;&lt;a href=&quot;http://backbonejs.org&quot;&gt;Backbone.js&lt;/a&gt; is a frontend library. Frontend means HTML. Frontend often means &lt;a href=&quot;http://jquery.org&quot;&gt;JQuery&lt;/a&gt; too. And besides application dependencies and view templates, you will load the application code, ideally in a &lt;a href=&quot;http://thinkingonthinking.com/what-is-modular-architecture/&quot;&gt;modular fashion&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;This might be very confusing if you are new to JavaScript or browser applications. To give you some faster start,  let’s write a simple seed project for a browser drawing app with Backbone.js, hmm… better: &lt;a href=&quot;http://backbonesvg.divshot.io/&quot;&gt;An SVG rectangle coloring app&lt;/a&gt; since it will be enough to show what Backbone is about.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/drawing_app.png&quot; /&gt;&lt;/p&gt;

&lt;h1 id=&quot;the-indexhtml&quot;&gt;The index.html&lt;/h1&gt;

&lt;p&gt;To start with browser interactions, you need some basic HTML. A simple HTML “gestalt” may look like:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;lt;html&amp;gt;
  &amp;lt;head&amp;gt;
  &amp;lt;/head&amp;gt;
  &amp;lt;body&amp;gt;
  &amp;lt;/body&amp;gt;
&amp;lt;/html&amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This HTML is the entry point for an application. You can instruct the browser to load and execute code by adding “script” tags.&lt;/p&gt;

&lt;p&gt;First, you must include JavaScript libs, i.e. Backbone and its dependencies &lt;a href=&quot;http://underscorejs.org&quot;&gt;Underscore&lt;/a&gt; and jQuery… If you have a network connection, the simplest way is including some references to CDNs entries:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;  &amp;lt;script src=&quot;https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.3/jquery.js&quot;&amp;gt;
  &amp;lt;/script&amp;gt;
  &amp;lt;script src=&quot;https://cdnjs.cloudflare.com/ajax/libs/underscore.js/1.5.2/underscore-min.js&quot;&amp;gt;
  &amp;lt;/script&amp;gt;
  &amp;lt;script src=&quot;https://cdnjs.cloudflare.com/ajax/libs/backbone.js/1.1.0/backbone-min.js&quot;&amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Those libs are rather small and may be included in the HTML header above. Note: JavaScript modules and dependeny managers to bundle external dependencies are quickly important to &lt;a href=&quot;http://thinkingonthinking.com/organizing-a-backbone-application/&quot;&gt;help you organize code&lt;/a&gt;. But for a fast start, you can skip this.&lt;/p&gt;

&lt;p&gt;With these libraries in place, we are now set to actually build a Backbone app!&lt;/p&gt;

&lt;h1 id=&quot;building-the-data-layer&quot;&gt;Building the Data layer&lt;/h1&gt;

&lt;p&gt;The next design step is about adding dummy data for a drawing. This may conflict with your instinct of adding SVG shapes directly. But separting a UI from “business logic” will save you some headaches in the long run.&lt;/p&gt;

&lt;p&gt;So, here is the data:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var drawing = [
  {id: 1, x1: 20, y1: 20, x2: 100, y2: 100, color: &apos;#ff0000&apos;}, 
  {id: 2, x1: 420, y1: 220, x2: 500, y2: 400, color: &apos;#00ff00&apos;}
];
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;By starting with data first has the advantage that you get a feeling for the abstractions you are working with. You don’t build a car from the outside in, but from the inside out based on an engine. Similarly, the data is the engine of your UI.&lt;/p&gt;

&lt;p&gt;If you are still thinking DOM nodes are your thing, no worries.  We come to that in a second.&lt;/p&gt;

&lt;p&gt;First, let’s wrap the raw drawing data in a Backbone &lt;a href=&quot;http://backbonejs.org/#Collection&quot;&gt;collection&lt;/a&gt; and &lt;a href=&quot;http://backbonejs.org/#Model&quot;&gt;models&lt;/a&gt;. With the Backbone API’s for data, it is easy to change and observe data changes.&lt;/p&gt;

&lt;p&gt;For the coloring app, the basic setup for Backbone models and collection is the following:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var Rectangle = Backbone.Model.extend({});
  
var Rectangles = Backbone.Collection.extend({
  model: Rectangle
});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;In your JavaScript, you can now work with instances of rectangles as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var rectangles = new Rectangles(drawing);
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;In principle, you could now change properties of the rectangles. You also could sort and filter the rectangles with native Backbone commands. For this, you can consult a book (ideally, the Pipefishbook).&lt;/p&gt;

&lt;h1 id=&quot;rendering-the-ui&quot;&gt;Rendering the UI&lt;/h1&gt;

&lt;p&gt;The fun of UI’s is in rendering. And, in capturing events from users. This is where the other half of Backbone.js comes in. For example, rendering rectangles could be done with Backbone.View as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;   // the scene class extends a Backbone view
   var Scene  = Backbone.View.extend({

     // renders an SVG rectangle from a shapes collection
     render: function() {
       var el = this.$el;
       el.empty();
       this.collection.each(function(model) {
         var rect = document.createElementNS(&quot;http://www.w3.org/2000/svg&quot;, &quot;rect&quot;);
         rect.setAttribute(&apos;x&apos;, model.get(&apos;x1&apos;));
         rect.setAttribute(&apos;y&apos;, model.get(&apos;y1&apos;));
         rect.setAttribute(&apos;width&apos;,  model.get(&apos;x2&apos;) - model.get(&apos;x1&apos;));
         rect.setAttribute(&apos;height&apos;, model.get(&apos;y2&apos;) - model.get(&apos;y1&apos;));
         rect.setAttribute(&apos;style&apos;, &apos;fill:&apos; + model.get(&apos;color&apos;) + &apos;;stroke:pink&apos;);
         rect.id = model.get(&apos;id&apos;);
         el.append(rect);
       });
       return el;
     }

   })
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Within the render function, you normally would call a template function. For learning, constructing a DOM node “manually” just works fine.&lt;/p&gt;

&lt;p&gt;Note, rendering happens in a context, often directly after the page load event:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;  var scene;
  $(document).ready(function() {
    scene = new Scene({el: $(&apos;svg&apos;), collection: rectangles });
  });
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h1 id=&quot;capturing-user-events&quot;&gt;Capturing user events&lt;/h1&gt;

&lt;p&gt;Next, and last, you want to offer some way to have user influence the rendered objects. For this, let’s hook into the mouse events of a user. A Backbone.View provides a property “events” for this.&lt;/p&gt;

&lt;p&gt;Working with events can quickly become complicated. To kepp it simple, let’s show an approach to handle click events. This gives:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;   var Scene  = Backbone.View.extend({

     events: {
       &apos;click rect&apos;: &apos;handleClick&apos;
     },

     handleClick: function(ev) {
       var id = $(ev.currentTarget).attr(&apos;id&apos;);
       var rect = this.collection.get(id);
       rect.set(&apos;color&apos;, getRandomColor());
     },

     // ...
   
     // binding events to data changes is important too:
    initialize: function() {
      this.listenTo(this.collection, &apos;change:color&apos;, this.render);
    }

  });
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;You see, the app size slowly increases. There is even new function getRandomColor(). But the main point right now: We capture events from users, and bind a view provides to change events in the data layer. The view re-renders as needed.&lt;/p&gt;

&lt;h1 id=&quot;going-further&quot;&gt;Going further&lt;/h1&gt;

&lt;p&gt;Because we have the conceptual model of the UI in Backbone collections and models, you can set the color from the browsers from other sources, than from user events, remote or from development console for example. Try the development console, open it, and try:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; rectangles
&amp;gt; r = rectangles.get(1)
&amp;gt; r.set(&apos;color&apos;, &apos;black&apos;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now, the color of the rectangles changes. Well, a “real” drawing app takes much more work. And you will be faced to re-use ideas and code for drawing circles or lines. Even, the simple function getRandomColor() is just copy and pasted from &lt;a href=&quot;http://stackoverflow.com/questions/1484506/random-color-generator-in-javascript&quot;&gt;here&lt;/a&gt;. Ideally, you could make code re-use easier. It is here where JavaScript module formats come in, such as CommonJS or RequireJS. Consult the &lt;a href=&quot;http://pipefishbook.com&quot;&gt;Pipefishbook&lt;/a&gt; for more info’s or leave me feedback. Thanks for reading!&lt;/p&gt;

&lt;p&gt;Checkout the &lt;a href=&quot;http://backbonesvg.divshot.io/&quot;&gt;demo&lt;/a&gt;.  Or the small &lt;a href=&quot;https://github.com/pipefishbook/rectangle_coloring&quot;&gt;repo&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;conflicts with the concept of a&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Rendering with React and JSX</title>
   <link href="https://thinkingonthinking.com/the-rendering-question/"/>
   <updated>2014-10-18T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/the-rendering-question</id>
   <content type="html">&lt;p&gt;To build interactive web applications and UIs in general, a lot of thinking goes into &lt;em&gt;rendering&lt;/em&gt; of items on a screen.&lt;/p&gt;

&lt;p&gt;For example, when I started programming with a C64 and later with MS-DOS, rendering pixels was about poking and peeking bytes for graphics chips. Besides shifting bits for pixels in an endless loop, you had to watch interrupts that were sayng, “Hey, a user event is waiting.”&lt;/p&gt;

&lt;h1 id=&quot;re-render-everything&quot;&gt;Re-render everything&lt;/h1&gt;

&lt;p&gt;With web applications today, as you can read in &lt;a href=&quot;http://pipefishbook.com/&quot;&gt;my book&lt;/a&gt;, rendering and interactions often happen by observing changes in key-value pairs in JavaScript, and updating the browser’s DOM accordingly.&lt;/p&gt;

&lt;p&gt;Yet with increasing levels of interactions, things stay difficult. Why should we observe large state objects and update manually the DOM? Why not, having a computer re-render everything, all the time?&lt;/p&gt;

&lt;p&gt;As &lt;a href=&quot;https://www.youtube.com/watch?v=GW0rj4sNH2w&quot;&gt;Jordan Walke explains in this talk&lt;/a&gt; at JSconf 2013, this is what React does. React knows the state of pixels in the browser &lt;em&gt;at any time&lt;/em&gt;. As such, you can stop worrying about events from users, and conflicting DOM updates, and have React &lt;a href=&quot;http://thinkingonthinking.com/key-value-pairs/&quot;&gt;synchronize the DOM with data over time&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Let’s take a short break from figuring out how React exactly works and let’s get started with rendering something with React. Take a look at &lt;a href=&quot;http://jsx-movies.divshot.io/&quot;&gt;this demo&lt;/a&gt; to render a movie with title and a small picture. To follow the code example, you might want to check out &lt;a href=&quot;https://github.com/pipefishbook/jsx-movies&quot;&gt;its repo here&lt;/a&gt;.&lt;/p&gt;

&lt;h1 id=&quot;try-it-out&quot;&gt;Try it out&lt;/h1&gt;

&lt;p&gt;Here are 5 simple steps to get some basic React foundations:&lt;/p&gt;

&lt;h2 id=&quot;1-install-basic-react-tools&quot;&gt;1) Install basic react tools&lt;/h2&gt;

&lt;p&gt;React comes with a number of &lt;a href=&quot;https://www.npmjs.org/package/react-tools&quot;&gt;command-line tools&lt;/a&gt;. One tool is especially helpful to transform the React JSX into JavaScript. Let’s install this with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ npm install react-tools
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;We look at the JSX syntax next, but in short: JSX allows you to declaratively tell what a view should look like and bind events to it.&lt;/p&gt;

&lt;h2 id=&quot;2-build-a-basic-view&quot;&gt;2) Build a basic view&lt;/h2&gt;

&lt;p&gt;Say, in your single-page application, you want to provide a nice UI for showing details of a movie. So, you need to define a &lt;em&gt;render&lt;/em&gt; method in a React view class. A basic view might look as follows in a file ./app/movie.jsx:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;/** @jsx React.DOM */

var MovieView = React.createClass({

  render: function() {
    var classString = &apos;movie&apos;;
    return (
      &amp;lt;article className={classString}&amp;gt;The Artist&amp;lt;/article&amp;gt;
    );

   }
});

module.exports = MovieView;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Besides the render method, you have a small comment on top that says, this view is not JavaScript but JSX. The extended JavaScript comes when you look at the return value of the render method. This is very close to actual HTML, together with the JSX syntax for a CSS class.&lt;/p&gt;

&lt;p&gt;By looking at the JSX, you can get a feeling for React’s model for rendering. You tell React, &lt;em&gt;what&lt;/em&gt; you expect from rendering a view. And this definition is valid at any point in time. You &lt;em&gt;don’t&lt;/em&gt; say &lt;em&gt;how&lt;/em&gt; the view should be rendered (which DOM nodes should be replaced), but &lt;em&gt;what&lt;/em&gt; the outcome should be.&lt;/p&gt;

&lt;p&gt;Behind the scenes, React manages differences from actual vs. needed view updates. As consequence, you let React decide &lt;em&gt;when&lt;/em&gt; a view must be updated.&lt;/p&gt;

&lt;p&gt;This may sound complicated for now, but it makes sense as you will shortly see. First, let’s prepare an environment to explore the view in the browser.&lt;/p&gt;

&lt;h2 id=&quot;2-prepare-browserify&quot;&gt;2) Prepare Browserify&lt;/h2&gt;

&lt;p&gt;The above view could be translated with the “jsx” command, but for browser environments I like my code to be wrapped as CommonJS modules.&lt;/p&gt;

&lt;p&gt;This is where &lt;a href=&quot;http://thinkingonthinking.com/unix-in-the-browser/&quot;&gt;browserify&lt;/a&gt; comes in. And, you can simply “require” JavaScript modules when and where needed.&lt;/p&gt;

&lt;p&gt;To make working with a view dependency as JavaScript module easier, the &lt;a href=&quot;https://www.npmjs.org/package/reactify&quot;&gt;reactify&lt;/a&gt; transform will become helpful sooner or later. I find this setup especially interesting for building &lt;a href=&quot;http://thinkingonthinking.com/javascript-matters-for-prototypes/&quot;&gt;UI prototypes&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;So, let’s install reactify, the JSX transformer for Browserify:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ npm install --save reactify
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;To see how this setup can matter for early prototyping, you can go to the &lt;a href=&quot;http://jsx-movies.divshot.io/&quot;&gt;live example&lt;/a&gt;. If you open the browser console, you can grab the reference to the rendered view and explore its state in the browser.&lt;/p&gt;

&lt;p&gt;Try:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; rendered.setState({isSelected: true})
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And when the view is selected, you can un-select the view with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; rendered.setState({isSelected: false});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;We will come back to what happens here shortly.&lt;/p&gt;

&lt;h2 id=&quot;3-browserify-a-view&quot;&gt;3) Browserify a view&lt;/h2&gt;

&lt;p&gt;The step needed to package a view module with Browserify comes next.&lt;/p&gt;

&lt;p&gt;The “normal” browserify command to package the view would be:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ browserify -t reactify -r ./app/view.jsx:view &amp;gt; static/view.js
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Yet, instead of going into &lt;a href=&quot;http://thinkingonthinking.com/intro-to-gulp/&quot;&gt;Gulp&lt;/a&gt; or Grunt discussions, let’s use a Makefile to automate view assembly:&lt;/p&gt;

&lt;p&gt;.PHONY: view&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;view:
	 browserify -t reactify -r ./app/movie.jsx:view &amp;gt; static/view.js
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now, whenever you run “make view”, you get a freshly assembled view ready for browser usage.&lt;/p&gt;

&lt;h2 id=&quot;4-bind-user-events&quot;&gt;4) Bind user events&lt;/h2&gt;

&lt;p&gt;No user would interact with JavaScript modules in a browser console. Users intuitively click or touch something on the rendered surface, or press keys on the keyboard to signal a change.&lt;/p&gt;

&lt;p&gt;Binding events to a JSX view happens directly with instructions on React’s virtual DOM nodes in the render method. For example, capturing a click event is done by adding an “onClick” handler. For the view in app/movie.jsx, this might look as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; /** @jsx React.DOM */

var MovieView = React.createClass({

  getInitialState: function() {
    return {isSelected: false}
  },

  select: function() {
    var isSelected = !this.state.isSelected;
    this.setState({isSelected: isSelected});
  },

  render: function() {
    console.log(this.state.isSelected);
    var selected = this.state.isSelected ? &apos;selected&apos; : &apos;&apos;;
    var classString = &apos;movie &apos; + selected;
    return (
      &amp;lt;article className={classString} onClick={this.select} onTouchStart={this.select}&amp;gt;The Artist&amp;lt;/article&amp;gt;
    );

   }
});

module.exports = MovieView;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;As you see, the render function now references the &lt;em&gt;state&lt;/em&gt; of the view. State is important for React to let it figure out, how a view should look like.&lt;/p&gt;

&lt;p&gt;Behind the scenes, you can think of views being rendered an infinite number of times with different states at different points in time.&lt;/p&gt;

&lt;p&gt;Since the rendering loop must start somewhere, the initial state of a view is often set manually, as you can see above in “getInitialState”.&lt;/p&gt;

&lt;p&gt;The state of the view can be modified with the “onClick” handler defined with “select: function() { … }”.&lt;/p&gt;

&lt;p&gt;Note, that for touch devices, events are handled separately with onTouchStart.&lt;/p&gt;

&lt;h2 id=&quot;5-assemble-the-html-and-deploy&quot;&gt;5) Assemble the HTML and deploy&lt;/h2&gt;

&lt;p&gt;Last, you can add some static HTML and CSS to make the interaction more fun. Also, it is might be interesting for you learn how to load React from HTML together with your browserified view.&lt;/p&gt;

&lt;p&gt;Note, that this setup is not a production setup, where you typically need to minify and optimize static assets for transport via HTTP.&lt;/p&gt;

&lt;p&gt;In the index.html, you first load the React library and then the view:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;lt;script src=&quot;react.js&quot;&amp;gt;&amp;lt;/script&amp;gt;
&amp;lt;script src=&quot;view.js&quot;&amp;gt;&amp;lt;/script&amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Since you browserified the code with the “-r” option in step 2), you can now “require” the React lib and the Movie view:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;lt;script&amp;gt;
  var React = require(&apos;react&apos;);
  React.initializeTouchEvents(true);

  var movie = require(&apos;view&apos;);
  var rendered = React.renderComponent(movie(), document.getElementById(&apos;app&apos;));
&amp;lt;/script&amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Besides “requiring” the React lib and movie view, we also mount the view to the “real” DOM. This happens above with the renderComponent instruction. And, you need to provide a DOM node, too.&lt;/p&gt;

&lt;p&gt;The whole setup is &lt;a href=&quot;https://github.com/pipefishbook/jsx-movies&quot;&gt;at Github&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;conclusions&quot;&gt;Conclusions&lt;/h2&gt;

&lt;p&gt;The rendering model of React feels great. It is certainly not like any of the other MVC libraries.&lt;/p&gt;

&lt;p&gt;Yet, if you want to increase the level of interactions in web pages, and the time that user stay on the same page (think Facebook), the React approach makes much sense.&lt;/p&gt;

&lt;p&gt;Mastering the tools and the syntax around JSX obviously takes some time obviously, but I hope that I gave you some introductory tips with this blog post. Hopefully, I will find enough time and feedback to explore this further.&lt;/p&gt;

&lt;p&gt;Please let me know what you like here, or take a look &lt;a href=&quot;http://pipefishbook.com/&quot;&gt;at the Pipefishbook&lt;/a&gt; to read more on Browserify or full-stack JavaScript.&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>JavaScript matters for prototypes</title>
   <link href="https://thinkingonthinking.com/javascript-matters-for-prototypes/"/>
   <updated>2014-10-09T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/javascript-matters-for-prototypes</id>
   <content type="html">&lt;p&gt;Building &lt;a href=&quot;http://thinkingonthinking.com/visual-languages/&quot;&gt;visual languages&lt;/a&gt; are key for reaching the goals of digital products. For web apps, visual language is woven into &lt;a href=&quot;http://www.nngroup.com/articles/flat-vs-deep-hierarchy/&quot;&gt;page navigation&lt;/a&gt;, &lt;a href=&quot;https://signalvnoise.com/posts/454-why-most-copywriting-on-the-web-sucks&quot;&gt;the content&lt;/a&gt; or the transitions in between.&lt;/p&gt;

&lt;p&gt;Why would you, as PHP or Ruby developer, want to start with JavaScript and Node, &lt;em&gt;before&lt;/em&gt; doing “cleaner” OOP design?&lt;/p&gt;

&lt;p&gt;Answering this question was &lt;a href=&quot;https://speakerdeck.com/mulderp/minimum-viable-interactions&quot;&gt;the talk I promised to deliver at Frontend Zurich&lt;/a&gt;. Unfortunately, I did not quite get there back then.&lt;/p&gt;

&lt;p&gt;Sorry if you were in the audience of my talk. You were probably wondering what the backend of &lt;a href=&quot;http://voki.me/&quot;&gt;a half-baked startup idea&lt;/a&gt; has to do with frontend technology?&lt;/p&gt;

&lt;p&gt;The message was: HTML and JavaScript is a great medium &lt;a href=&quot;http://blog.ted.com/2010/05/04/how_great_leade/&quot;&gt;to inspire action&lt;/a&gt;.  And, with &lt;a href=&quot;http://nobackend.org/&quot;&gt;nobackends&lt;/a&gt; it has never been easier.&lt;/p&gt;

&lt;p&gt;This can be seen from the following 10 steps, from zero to working prototype of &lt;a href=&quot;http://burners.divshot.io/&quot;&gt;a goal sharing prototype&lt;/a&gt;.&lt;/p&gt;

&lt;h1 id=&quot;0--goals-of-the-app&quot;&gt;0)  Goals of the app&lt;/h1&gt;

&lt;p&gt;Without structure, a basic idea outline, it will be hard to arrive somewhere. So, the goals of the &lt;a href=&quot;http://burners.divshot.io/&quot;&gt;burners&lt;/a&gt; app are:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;define your goals for the next X weeks&lt;/li&gt;
  &lt;li&gt;share the goals with someone else via a web browser&lt;/li&gt;
  &lt;li&gt;provide basic functions for setting priorities&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Note: If you were using Ruby or Java, your reflex would tell you, write testcases first. But for this blog post, I want to &lt;em&gt;skip&lt;/em&gt; writing tests. The main test of the app will be whether a user &lt;em&gt;grasps its purpose&lt;/em&gt;.&lt;/p&gt;

&lt;h1 id=&quot;1--init-divshot&quot;&gt;1)  Init divshot&lt;/h1&gt;

&lt;p&gt;To get some raw HTML, I like:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ divshot init
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;&lt;a href=&quot;https://www.npmjs.org/package/divshot&quot;&gt;divshot&lt;/a&gt; is a command -line tool that - besides providing a basic project structure - allows you to continuously deploy a static page from the command line.&lt;/p&gt;

&lt;h1 id=&quot;2--init-npm&quot;&gt;2)  Init npm&lt;/h1&gt;

&lt;p&gt;For re-using code, you want to make use &lt;a href=&quot;http://www.modulecounts.com/&quot;&gt;of the world’s largest code repository&lt;/a&gt;, don’t you? So, before going further, make sure you have a valid package.json. One way to do this, is:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ npm init
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h1 id=&quot;3--get-your-favorite-js-libs&quot;&gt;3)  Get your favorite JS libs&lt;/h1&gt;

&lt;p&gt;Next, you could use &lt;a href=&quot;http://yeoman.io/&quot;&gt;yeoman&lt;/a&gt; to fetch some basic libraries and a project structure. Actually, I am quicker by hand for simple prototypes:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ npm install --save -d underscore jquery-untouched backbone
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;With these libs, you can easily handle browser events, DOM manipulations, as well as run &lt;a href=&quot;https://developer.mozilla.org/en-US/docs/AJAX/Getting_Started&quot;&gt;XMLHttpRequest (or Ajax)&lt;/a&gt; requests.&lt;/p&gt;

&lt;h1 id=&quot;4--setup-a-build-process&quot;&gt;4)  Setup a build process&lt;/h1&gt;

&lt;p&gt;Browser apps are special in that sense, that they require developers to think, &lt;a href=&quot;https://vimeo.com/62988591&quot;&gt;how to package an app&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;In this example, I chose a Makefile and &lt;a href=&quot;https://github.com/substack/node-browserify&quot;&gt;browserify (see next step)&lt;/a&gt;. Not much, is simpler.&lt;/p&gt;

&lt;h1 id=&quot;5-setup-browserify&quot;&gt;5) Setup Browserify&lt;/h1&gt;

&lt;p&gt;I like the idea of sharing structures between browser and server &lt;a href=&quot;http://thinkingonthinking.com/scripting-a-csv-converter/&quot;&gt;REPL environments&lt;/a&gt;.  To sort and poke with your goals, you want to write something along:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; goals = require(&apos;collections/goals&apos;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The same as on the server! And, you can experiment directly with your goals in a console, (e.g. extend sort logic), no matter where.&lt;/p&gt;

&lt;h1 id=&quot;6-templates&quot;&gt;6) Templates&lt;/h1&gt;

&lt;p&gt;Much of communication with users, relies on good putting data into templates. I like &lt;a href=&quot;https://github.com/wycats/handlebars.js/&quot;&gt;handlebars&lt;/a&gt;, and with &lt;a href=&quot;https://github.com/FormidableLabs/handlebones&quot;&gt;handlebones&lt;/a&gt;, it combines great with Backbone.js.&lt;/p&gt;

&lt;h1 id=&quot;7-backend-and-admin-interface-with-firebase&quot;&gt;7) Backend and admin interface with Firebase&lt;/h1&gt;

&lt;p&gt;Compared to the previous steps, it is here where we can leverage most to get the prototype working. A number of &lt;a href=&quot;https://blog.engineyard.com/2014/backend-as-a-service&quot;&gt;backend as a services&lt;/a&gt; are interesting. Especially, I think the &lt;a href=&quot;https://www.arangodb.org/foxx&quot;&gt;open-source ArangoDB and Foxx&lt;/a&gt; have a nice future. But staying with today, as first step, let’s take &lt;a href=&quot;https://www.firebase.com/&quot;&gt;Firebase&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;On a sidenote, you can use Firebase later for a basic admin service too. See &lt;a href=&quot;/build-your-email-list-with-firebase&quot;&gt;Firebase and Backbone&lt;/a&gt; for more information here.&lt;/p&gt;

&lt;h1 id=&quot;8-wire-up-the-backend&quot;&gt;8) Wire up the backend&lt;/h1&gt;

&lt;p&gt;If you use backbone, you basically wire up Backbone collections to Firebase.  You can find some directions on this in &lt;a href=&quot;http://pipefishbook.com&quot;&gt;my book&lt;/a&gt; or at its &lt;a href=&quot;https://github.com/pipefishbook/ch_7/tree/master/firebase&quot;&gt;code repository&lt;/a&gt;.&lt;/p&gt;

&lt;h1 id=&quot;9-fill-up-the-html-and-css&quot;&gt;9) Fill up the HTML and CSS&lt;/h1&gt;

&lt;p&gt;So far, we mainly stayed with JS and some build processes. It is time to add more static assets, such as HTML and some basic CSS. (Note: I am not a CSS wizard, so, feel free to comment how to make my CSS better.)&lt;/p&gt;

&lt;h1 id=&quot;10-enjoy&quot;&gt;10) Enjoy!&lt;/h1&gt;

&lt;p&gt;Finished. By setting the access rights for your data at Firebase, you can easily change who can read and write goals. The goals I had before summer are public now. &lt;a href=&quot;http://burners.divshot.io/&quot;&gt;Take a look!&lt;/a&gt;. Did I reach my goals in the meanwhile? Well, it depends. I found out what will matter most to me. And, more important, I did not lose (too much) time, with writing code, I probably won’t need anyway.&lt;/p&gt;

&lt;p&gt;So, the code repository is &lt;a href=&quot;https://github.com/mulderp/burners&quot;&gt;here&lt;/a&gt;, rather sandbox like and without too much documentation. If you want to learn more on Node.js or Backbone.js, I suggest you take a look &lt;a href=&quot;http://pipefishbook.com/&quot;&gt;at my book&lt;/a&gt;.&lt;/p&gt;

&lt;bold&gt;Edit:&lt;/bold&gt;
&lt;p&gt;This post got some &lt;a href=&quot;http://www.reddit.com/r/javascript/comments/2iqrb3/javascript_matters_for_prototyping/cl4ogg2&quot;&gt;nice feedback&lt;/a&gt; at Reddit. Espeically, &lt;a href=&quot;https://www.npmjs.org/package/insert-css&quot;&gt;insert-css&lt;/a&gt; looks nice!&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Switching Blogs</title>
   <link href="https://thinkingonthinking.com/switching-blogs/"/>
   <updated>2014-07-18T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/switching-blogs</id>
   <content type="html">&lt;p&gt;July. Around 4 months of freelancing. Time to look back.&lt;/p&gt;

&lt;p&gt;While I had certain ideas in mind on &lt;a href=&quot;http://thinkingonthinking.com/customers-jobs/&quot;&gt;possible software projects&lt;/a&gt;, things turned out a bit different in the meantime.&lt;/p&gt;

&lt;p&gt;As I &lt;a href=&quot;http://thinkingonthinking.com/customers-jobs/&quot;&gt;previously described&lt;/a&gt;, I expected to find customers with interests in distributed “research &amp;amp; development” or “open” source software.&lt;/p&gt;

&lt;p&gt;But most software projects that I see focus on &lt;em&gt;processes&lt;/em&gt; like TDD or Scrum, or &lt;em&gt;frameworks&lt;/em&gt; like Rails or Angular. From both, technical and from &lt;a href=&quot;http://thinkingonthinking.com/first-week-as-freelancer/&quot;&gt;business model perspectives&lt;/a&gt;, I find these projects less interesting, since processes and frameworks constrain the vision on products. And, it is hard to see &lt;a href=&quot;http://thinkingonthinking.com/Elegance-in-a-software-business/&quot;&gt;elegance&lt;/a&gt; when the user of the software is unclear, or the problem the software should solve, or how to make money from the digital product you should build.&lt;/p&gt;

&lt;p&gt;So, unfortunately, my first idea of supporting “research &amp;amp; development” projects has not yet worked out, at least &lt;em&gt;not&lt;/em&gt; in the context of developing &lt;em&gt;software&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;Yet living in Germany, and Bavaria especially, there is one kind of product that makes a number of companies successfull: Cars.&lt;/p&gt;

&lt;p&gt;First, the business models of cars is clear: Buy parts, assemble these, and sell the result at a higher price than the total costs of the parts. With the dawn of the electrical car and new mobility models, there are plenty of companies that invest in R&amp;amp;D currently. And while cars are all around us, the requirements on their parts are heavily changing.&lt;/p&gt;

&lt;p&gt;As an important side-effect, it is the automotive industry, where I found my first good, interesting freelance project.&lt;/p&gt;

&lt;p&gt;I am currently supporting a company with R&amp;amp;D of new car products. While it is not directly a software project, I can apply my ideas on software perfectly. Me and my team members are the users of my software products, the problems the software must solve are very clear too.&lt;/p&gt;

&lt;p&gt;Due to nature of my contract, the work is not open-source, at least not from the implementation point of view. When working with these large companies, specifications and protocols are partly open. But how the specs are achieved or the protocols are implemented, is part of the company secrets, and part of their business model.&lt;/p&gt;

&lt;p&gt;Luckily, with my learnings on writing &lt;a href=&quot;http://shop.oreilly.com/product/0636920030799.do&quot;&gt;a book for O’Reilly&lt;/a&gt; and a comment of &lt;a href=&quot;https://www.sparkfun.com/pages/Nathan&quot;&gt;Nathen Seidle&lt;/a&gt; on &lt;a href=&quot;https://news.ycombinator.com/item?id=8043157&quot;&gt;open-source hardware&lt;/a&gt;,  I got some fresh perspectives on how digital products can be built with an open-source approach, albeit hardware products.&lt;/p&gt;

&lt;p&gt;To document my learnings on hardware products and an open-source approach around it, I will switch the blog for a while. My new writings on hardware will be at: &lt;a href=&quot;http://blog.farsinotare.com&quot;&gt;blog.farsinotare.com&lt;/a&gt;. Since we are just at the beginning of connecting hardware devices, and there is much to learn, I also founded the &lt;a href=&quot;http://www.meetup.com/Munchen-Arduino-Meetup/&quot;&gt;Munich Arduino Meetup&lt;/a&gt;. &lt;a href=&quot;http://arduino.cc/&quot;&gt;Arduino&lt;/a&gt;, yet another great company that makes users happy with “open” products.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>CommonJS for the database</title>
   <link href="https://thinkingonthinking.com/commonjs-for-databases/"/>
   <updated>2014-06-13T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/commonjs-for-databases</id>
   <content type="html">&lt;p&gt;Thanks to &lt;a href=&quot;https://github.com/substack/node-browserify&quot;&gt;Browserify&lt;/a&gt;, the &lt;a href=&quot;http://wiki.commonjs.org/wiki/CommonJS&quot;&gt;CommonJS&lt;/a&gt; module format is spreading into the browser.  Since some months, you can also use CommonJS modules in the &lt;a href=&quot;https://www.arangodb.org/&quot;&gt;ArangoDB&lt;/a&gt; database. Why would this be interesting?&lt;/p&gt;

&lt;p&gt;Let’s start with a pattern: In many web applications, &lt;a href=&quot;http://www.martinfowler.com/eaaCatalog/activeRecord.html&quot;&gt;ActiveRecord&lt;/a&gt; objects are important building blocks to explore and query data. With specialized objects, you can easily enhance and organize data queries. And, with CommonJS modules, you can do this easily on top of raw &lt;a href=&quot;http://en.wikipedia.org/wiki/JSON&quot;&gt;JSON&lt;/a&gt; documents.&lt;/p&gt;

&lt;p&gt;Although a &lt;a href=&quot;https://www.arangodb.org/2012/11/13/comparing-arangodb-with-mongodb-and-couchdb&quot;&gt;number of document stores&lt;/a&gt; provide JSON support, ArangoDB and its support of CommonJS modules stand out. As the shell of ArangoDB is based on V8, you can use Arangosh in a similar way as you would run Node to execute JavaScript. But by running “arangosh”, you additionally have access to collections (and graphs) in the database.&lt;/p&gt;

&lt;p&gt;Let’s apply the idea of querying data with objects, with data coming from the &lt;a href=&quot;http://www.kimonolabs.com/worldcup/explorer&quot;&gt;Kimonolabs&lt;/a&gt; WorldCup football API. Hopefully, you’ll get a feeling for:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Exploring a data set with JavaScript and ArangoDB&lt;/li&gt;
  &lt;li&gt;Using CommonJS modules to encapsulate data and queries&lt;/li&gt;
&lt;/ul&gt;

&lt;h1 id=&quot;setup-with-json&quot;&gt;Setup with JSON&lt;/h1&gt;

&lt;p&gt;Since it is the time of the football worldcup, the Worldcup API of &lt;a href=&quot;&quot;&gt;Kimonolabs&lt;/a&gt; provides a nice data set to start with. After getting an API key, &lt;a href=&quot;https://www.kimonolabs.com/worldcup/docs#MatchObject&quot;&gt;football matches&lt;/a&gt; can be retrieved with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var request = require(&quot;request&quot;);
fs = require(&apos;fs&apos;);

request(&quot;http://worldcup.kimonolabs.com/api/matches?apikey=...&quot;, 
function(err, response, body) {
  fs.writeFile(&quot;matches.json&quot;, body);
});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Similarly, &lt;a href=&quot;https://www.kimonolabs.com/worldcup/docs#ListTeams&quot;&gt;teams&lt;/a&gt; can be retrieved with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var request = require(&quot;request&quot;);
fs = require(&apos;fs&apos;);

request(&quot;http://worldcup.kimonolabs.com/api/teams?apikey=...&quot;, 
function(err, response, body) {
  fs.writeFile(&quot;teams.json&quot;, body);
});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;After running these two API requests, you should have the files:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;matches.json
teams.json
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Inspecting the data, you will see that the football matches only contain references to the teams, but their names. Embedding the teams in a match object, will be our first task.&lt;/p&gt;

&lt;p&gt;In order to work with the data in ArangoDB, you must import the data with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ arangoimp --collection matches --create-collection true matches.json
$ arangoimp --collection teams --create-collection true teams.json
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h1 id=&quot;enhanced-data&quot;&gt;Enhanced Data&lt;/h1&gt;

&lt;p&gt;Let’s write our first CommonJS module that help querying matches. The basic syntax to define a CommonJS module is with the following syntax in a file collections/matches.js:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var Matches = function(raw) {
  this.raw = raw;
}
module.exports = Matches;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;You can then load the query class in the Arango shell:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;arangosh [_system]&amp;gt; var Matches = require(&apos;collections/matches&apos;)

arangosh [_system]&amp;gt; matches = new Matches(db.matches)
[object Object]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This collection class can’t do much yet. To have the module doing something useful, we extend collections/matches.js with a list function as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;Matches.prototype.list = function() {
  return this.raw.toArray();
}
module.exports = Matches;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now, in Arangosh, you can write:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;rangosh [_system]&amp;gt; matches.list()
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And, you’ll get something back that looks like:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;[ 
  { 
  &quot;_id&quot; : &quot;matches/753627501&quot;, 
  &quot;_rev&quot; : &quot;753627501&quot;, 
  &quot;_key&quot; : &quot;753627501&quot;, 
  &quot;currentGameMinute&quot; : null, 
  &quot;homeScore&quot; : 0, 
  &quot;awayScore&quot; : 0, 
  &quot;group&quot; : &quot;D&quot;, 
  &quot;type&quot; : &quot;Match&quot;, 
  &quot;startTime&quot; : &quot;2014-06-24T16:00:00.000Z&quot;, 
  &quot;status&quot; : &quot;Pre-game&quot;, 
  &quot;venue&quot; : &quot;Arena das Dunas (Natal, Rio Grande do Norte)&quot;, 
  &quot;awayTeamId&quot; : &quot;088C4113-CEFC-460C-830C-277C148C0CE7&quot;, 
  &quot;homeTeamId&quot; : &quot;B61B25AA-CD8E-4778-AC26-DD08D7851990&quot;, 
  &quot;id&quot; : &quot;79BBB6ED-7E6B-4239-A287-177F97F10E3F&quot; 
  }, 
  // ...
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;But as mentioned before, awayTeamId and homeTeamId are references to another collection, and we would like to see the actual teams in a match object, not the referenced ID.&lt;/p&gt;

&lt;p&gt;To achieve this goal, you can make a teams collection as follows in collections/teams.js:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var _ = require(&apos;underscore&apos;);

var Teams = function(raw) {
  this.raw = raw;
}

Teams.prototype.show = function(id) {
  return _.first(this.raw.byExample({id: id}).toArray());
}

module.exports = Teams;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Note how you can include the &lt;a href=&quot;http://underscorejs.org/&quot;&gt;Underscore&lt;/a&gt; module to take advantage of its enumeration helpers, such as “_.first()” on an Array.&lt;/p&gt;

&lt;p&gt;Next up, you can embed the teams in the Matches collection as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var _ = require(&apos;underscore&apos;);
var Teams = require(&apos;./teams&apos;);

var Matches = function(matches, teams) {
  this.raw = matches;
  this.teams = new Teams(teams);
}

Matches.prototype.show = function(id) {
  var match = _.first(this.raw.byExample({id: id}).toArray());
  match.awayTeam = this.teams.show(match.awayTeamId);
  match.homeTeam = this.teams.show(match.homeTeamId);

  return match;
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;In Arangosh, you write:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;arangosh [_system]&amp;gt; matches = new Matches(db.matches, db.teams)
arangosh [_system]&amp;gt; matches.show(&quot;BBBE6B39-E345-43C7-9E31-A442A866BF60&quot;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And, you should now see an example of a match with embedded teams:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;{ 
  &quot;_id&quot; : &quot;matches/752644461&quot;, 
  &quot;_rev&quot; : &quot;752644461&quot;, 
  &quot;_key&quot; : &quot;752644461&quot;, 
  &quot;currentGameMinute&quot; : null, 
  &quot;homeScore&quot; : 0, 
  &quot;awayScore&quot; : 0, 
  &quot;group&quot; : &quot;H&quot;, 
  &quot;type&quot; : &quot;Match&quot;, 
  &quot;id&quot; : &quot;8CF356AE-1F8D-4E61-B92A-A936F72B60B6&quot;, 
  &quot;awayTeam&quot; : { 
    // ...
    &quot;name&quot; : &quot;Korea Republic&quot;, 
    &quot;homeStadium&quot; : &quot;Seoul World Cup Stadium&quot;, 
    &quot;id&quot; : &quot;8D6EAC04-14E9-4026-BF2A-AB81C4F3C529&quot; 
  }, 
  &quot;homeTeam&quot; : { 
    // ...
    &quot;homeStadium&quot; : &quot;Olimpiyskiy stadion Luzhniki&quot;, 
    &quot;id&quot; : &quot;BBBE6B39-E345-43C7-9E31-A442A866BF60&quot; 
  } 
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;So, instead of writing a complex query with joins or subqueries, you now have some query object that wraps the relationship between Match and Teams.&lt;/p&gt;

&lt;p&gt;But there are more options of using JavaScript to explore data in ArangoDB.&lt;/p&gt;

&lt;h1 id=&quot;wrapping-advanced-queries&quot;&gt;Wrapping advanced queries&lt;/h1&gt;

&lt;p&gt;For advanced queries, ArangoDB provides AQL, an NoSQL query language that has some similarities with SQL. I wrote about some AQL basics in &lt;a href=&quot;http://thinkingonthinking.com/aql-for-twitter-analysis/&quot;&gt;a previous post&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;In this example, let’s write an AQL query to filter matches for a given date:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;FOR m IN matches
  FILTER DATE_TIMESTAMP(m.startTime) &amp;gt; DATE_TIMESTAMP(&quot;2014-06-14&quot;)
  FILTER DATE_TIMESTAMP(m.startTime) &amp;lt; DATE_TIMESTAMP(&quot;2014-06-16&quot;)
  RETURN m.startTime
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;We can easily add this query to the Matches query class as follows. In collection/matches:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;Matches.prototype.today = function() {
  var todayFilter = &apos;for m in matches \
  filter DATE_TIMESTAMP(m.startTime) &amp;gt; DATE_TIMESTAMP(&quot;2014-06-14&quot;) \
  filter DATE_TIMESTAMP(m.startTime) &amp;lt; DATE_TIMESTAMP(&quot;2014-06-16&quot;) \
    return m&apos;;

  return db._query(todayFilter).toArray();
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Running the following in Arangosh:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;arangosh [_system]&amp;gt; matches.today()
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Results in:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;[ 
  { 
    &quot;_id&quot; : &quot;matches/754413933&quot;, 
    &quot;homeScore&quot; : 3, 
    &quot;awayScore&quot; : 0,
    // ... 
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Another nice thing on JavaScript query classes: You can include most of the 70k JavaScript modules from &lt;a href=&quot;https://www.npmjs.org/&quot;&gt;npm&lt;/a&gt; to support your queries. For example, &lt;a href=&quot;https://www.npmjs.org/package/moment&quot;&gt;moment&lt;/a&gt; to format and parse date types as needed.&lt;/p&gt;

&lt;h1 id=&quot;conclusions&quot;&gt;Conclusions&lt;/h1&gt;

&lt;p&gt;Using CommonJS in a database to build query classes looks very promising. You can enhance the basic JSON key-value pairs coming from a document store. With ArangoDB, you can organize advanced AQL queries. You could even serve your queries via HTTP and Foxx to the outside world.&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Usability is Everywhere</title>
   <link href="https://thinkingonthinking.com/Usability-is-Everywhere/"/>
   <updated>2014-06-05T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/Usability-is-Everywhere</id>
   <content type="html">&lt;blockquote class=&quot;twitter-tweet&quot; lang=&quot;en&quot;&gt;
In this guest post, &lt;a href=&quot;http://www.futureground.net/?utm_source=totmulder&amp;amp;utm_medium=web&amp;amp;utm_campaign=uie201406&quot;&gt;Jeff Bellsey&lt;/a&gt; explains what makes an excellent user experience.  Discovering respect and empathy may be hard for most techies, but the &quot;divine spark&quot; is an old concept and often overlooked in our busy digital lifes. Read on to see how the conversation on a (non-) typical meetup talk resulted into a productive learning experience...
&lt;/blockquote&gt;

&lt;p&gt;The presentation was painful. You’ve been there too: the presenter is clearly knowledgeable, articulate, and well-respected. Yet somehow you’re writhing in your seat, bored and a bit annoyed, wondering when it will be over. Why? He failed to connect.&lt;/p&gt;

&lt;p&gt;I’ve been to about a dozen of these presentations in the last year. Public speaking is a different skill set than programming, we all know that. But that’s not what I’m talking about. See, we’ve also been to talks by presenters who were not terribly good speakers, although they were still excellent &lt;em&gt;communicators&lt;/em&gt;. They might have been uncomfortable on stage, but they had good pacing, clear goals, and a respect for your time and attention. Those qualities are available to anyone who strives for them, regardless of stage fright.&lt;/p&gt;

&lt;p&gt;Usability is everywhere. It’s not a thing you do when designing information architecture or storyboards or interaction cues. Usability isn’t a discrete concept or object or discipline. Usability is an &lt;em&gt;ethic&lt;/em&gt;, and you can’t avoid it.&lt;/p&gt;

&lt;h2 id=&quot;ethics&quot;&gt;Ethics&lt;/h2&gt;

&lt;p&gt;Human beings are always in relationship. We’re always bumping into each other, affecting each other, influencing and supporting and worrying about each other. Almost &lt;em&gt;everything&lt;/em&gt; you do has an effect on other people. This one truth — simple, but whoa-big — is the foundation of ethics.&lt;/p&gt;

&lt;p&gt;I remember the ton-of-bricks experience of letting this sink in. I started doing everything differently. My code commenting took a huge leap forward, now that I cared about the programmer who would later have to figure out my magic tricks. I wanted them to feel guided, not lost or impressed. Even before I met him, I was in relationship with the next guy. My error messages became gentler, my screen layouts more spacious, my documentation more task-driven.&lt;/p&gt;

&lt;p&gt;Ethics was something I had been thinking about a lot early in my career, as part of a philosophical orientation towards life. It was &lt;a href=&quot;http://www.amazon.com/gp/product/1568843224/ref=as_li_tl?ie=UTF8&amp;amp;camp=1789&amp;amp;creative=390957&amp;amp;creativeASIN=1568843224&amp;amp;linkCode=as2&amp;amp;tag=goitogro-20&amp;amp;linkId=7YMD5VKVRJ3PCPYV&quot;&gt;Alan Cooper’s About Face&lt;/a&gt; that made the all-important connection between this broad inquiry and my vocation. If you haven’t read his books on usability yet, you owe it to yourself (and to your users!) to get them. He has deep insight into this issue.&lt;/p&gt;

&lt;p&gt;Jason Fried of 37signals &lt;a href=&quot;http://signalvnoise.com/posts/454-why-most-copywriting-on-the-web-sucks&quot;&gt;said&lt;/a&gt; it very succinctly: “Most copywriting on the web sucks because it’s written for the writer, not for the reader. Write for the reader. That is all.” Your job is to extend that attitude to everything.&lt;/p&gt;

&lt;h2 id=&quot;down-to-earth&quot;&gt;Down to earth&lt;/h2&gt;

&lt;p&gt;This might come as a shock, but people think differently than you do. We all have different mental models, different ways of digesting information. We have different histories, different experiences, different aspirations. This is the beauty of life’s complexity. And because we’re always in relationship, it’s one of life’s greatest challenges.&lt;/p&gt;

&lt;p&gt;When you teach or write code or design interfaces, are you conscious of this challenge? Are you aware of how people are receiving you? Or are you just dumping, forcing people to think the way you do (or assuming they already do)? What kind of contact are you making? Is your communication ethical, in the way I just described? Are you guiding and creating joy and insight, or adding to confusion, frustration, even cynicism? Are you showing respect and care, or just being lazy?&lt;/p&gt;

&lt;p&gt;There is no simple prescription in this post. You won’t find me saying something like “keep your presentations modest” or “always over-comment your code.” There’s nothing modest about &lt;a href=&quot;http://www.spacex.com/hyperloop&quot;&gt;Hyperloops&lt;/a&gt; or &lt;a href=&quot;http://www.cosmosontv.com/&quot;&gt;&lt;em&gt;Cosmos&lt;/em&gt;&lt;/a&gt; or &lt;a href=&quot;http://www.amazon.com/gp/product/067443000X/ref=as_li_tl?ie=UTF8&amp;amp;camp=1789&amp;amp;creative=390957&amp;amp;creativeASIN=067443000X&amp;amp;linkCode=as2&amp;amp;tag=goitogro-20&amp;amp;linkId=PDWF53HV5SU6HEUO&quot;&gt;&lt;em&gt;Capital in the Twenty-First Century&lt;/em&gt;&lt;/a&gt;, and that’s a good thing. With apologies to my forebears, there are no “Ten Commandments of Ethics” that have general applicability, no rules of thumb that guarantee you’ll make contact with your users and your audience.&lt;/p&gt;

&lt;p&gt;The real prescription is simpler and more challenging: you have to &lt;em&gt;care&lt;/em&gt;. If you aspire to greater care and empathy as a person and as a technologist, your presentations will connect, your code will self-document, your designs will delight. If you bring a passion for character development along with your love of learning and creating, your users will feel it.&lt;/p&gt;

&lt;p&gt;Usability is everywhere. Ethics are everywhere. The more care you show for your users, the more successful you will be.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Visual Languages for the Web</title>
   <link href="https://thinkingonthinking.com/visual-languages/"/>
   <updated>2014-06-01T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/visual-languages</id>
   <content type="html">&lt;p&gt;When HTML was first conceived in &lt;a href=&quot;http://en.wikipedia.org/wiki/HTML#Development&quot;&gt;1989&lt;/a&gt;, computers were dealing mainly with textual documents. In the meantime, we are dealing more and more with visual information. With web browsers, we are trying to solve the same problems that were solved with special spreadsheet, charting or slideshow software packages.&lt;/p&gt;

&lt;p&gt;Luckily, browsers are perfectly able to do stunning visualizations based on HTML, CSS and &lt;a href=&quot;http://en.wikipedia.org/wiki/Svg&quot;&gt;SVG&lt;/a&gt;. With the HTML5 standard, browsers also support a new &lt;a href=&quot;http://en.wikipedia.org/wiki/Canvas_element&quot;&gt;canvas&lt;/a&gt; tag.&lt;/p&gt;

&lt;p&gt;Unfortunately, looking at many scientific or engineering websites, most visual data comes as JPG or PNG, instead of code that can be interpreted or adapted as needed.&lt;/p&gt;

&lt;p&gt;Good places to explore visual ideas in browsers are &lt;a href=&quot;http://d3js.org/&quot;&gt;D3.js&lt;/a&gt; and &lt;a href=&quot;http://processingjs.org/&quot;&gt;ProcessingJS&lt;/a&gt;. Besides support from these general infoviz libraries, there are often specialized JavaScript libraries to solve visual problems. For example, to draw digital waveforms &lt;a href=&quot;https://github.com/drom/wavedrom&quot;&gt;wavedrom&lt;/a&gt; looks interesting. Or, to draw and interact with small logical circuits, &lt;a href=&quot;http://www.jointjs.com/demos/logic&quot;&gt;JointJS&lt;/a&gt; looks great. For rendering graphs, Andrei Kashcha’s &lt;a href=&quot;https://github.com/anvaka/ngraph&quot;&gt;ngraph&lt;/a&gt; project looks great and was recently featured in a &lt;a href=&quot;https://www.youtube.com/watch?v=8CX-Q0gtSp8&quot;&gt;TEDx Stanford&lt;/a&gt; talk.&lt;/p&gt;

&lt;p&gt;But besides learning the tricks of libraries, crafting visual languages is important. This requires you to explore encodings and play with stories around graphics. For example at Fluent Conference 2014, Scott Murray shows how &lt;a href=&quot;https://speakerdeck.com/alignedleft/interface-design-considerations-for-data-visualization&quot;&gt;visual interface design&lt;/a&gt; can be explored with D3 and the &lt;a href=&quot;https://www.youtube.com/watch?v=e1HMmLcnpXg&quot;&gt;value of process&lt;/a&gt;:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/design_rules.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;That a design process is important to explore data is also emphasized in Ben Fry’s &lt;a href=&quot;http://shop.oreilly.com/product/9780596514556.do&quot;&gt;book “Visualizing Data”&lt;/a&gt;, where different stages for visualization design are discussed.&lt;/p&gt;

&lt;p&gt;For my goals, the basic visualization goals would be:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;prepare datasets with JSON&lt;/li&gt;
  &lt;li&gt;visualize datasets&lt;/li&gt;
  &lt;li&gt;provide interactions&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The main power of D3 and Processing come in steps 2 and 3. Working with D3 and/or Processing can help you to explore and identify &lt;a href=&quot;http://thinkingonthinking.com/the-value-of-iteration/&quot;&gt;ideas that can stick&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Let’s compare both approaches by drawing a simple line.&lt;/p&gt;

&lt;h2 id=&quot;d3&quot;&gt;D3&lt;/h2&gt;

&lt;p&gt;In D3, you start with data that might look like:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; var dataset = [{coord: [10, 20, 30, 40]}];
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The main point is that coordinates are wrapped in some property of a data element. The power of D3 is the &lt;a href=&quot;http://alignedleft.com/tutorials/d3/binding-data&quot;&gt;“mapping” or “binding”&lt;/a&gt; of data to a visual language of shapes and colors. Besides data binding, the toolbox of D3 provides many abstractions and tricks.&lt;/p&gt;

&lt;p&gt;Another important idea of D3 is “chaining” commands. So, once you have bound data to properties of HTML or SVG tags, you can add more visual transformations, or add event bindings.&lt;/p&gt;

&lt;p&gt;Drawing a line with D3 might then look like:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;lt;svg&amp;gt;
  &amp;lt;g id=&quot;canvas&quot;&amp;gt;&amp;lt;/g&amp;gt;
&amp;lt;/svg&amp;gt;

&amp;lt;script type=&quot;text/coffeescript&quot;&amp;gt;
  console.log d3
  svg = d3.select(&apos;svg&apos;)
  svg.attr({height: 400, width: 500})
  svg.selectAll(&apos;line&apos;)
     .data(dataset)
     .enter()
     .append(&apos;line&apos;)
     .attr(&apos;stroke&apos;, &apos;#000&apos;)
     .attr(&apos;x1&apos;, (d) -&amp;gt; d.points[0])
     .attr(&apos;x2&apos;, (d) -&amp;gt; d.points[2])
&amp;lt;/script&amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Note the usage of &lt;a href=&quot;http://coffeescript.org/#scripts&quot;&gt;CoffeeScript in the browser&lt;/a&gt;. This is not something you would do in a production app, but for exploring ideas, CoffeeScript feels nice here. Also notice, how using D3 can abstract away the coordinate system easily. This might be good or bad, depending on your goals.&lt;/p&gt;

&lt;p&gt;D3 is also an important building projects in projects dealing with data management, such as charting libraries as &lt;a href=&quot;https://github.com/fastly/epoch&quot;&gt;Epoch&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;processingjs&quot;&gt;Processing.JS&lt;/h2&gt;

&lt;p&gt;The idea of visualizing things with Processing is a bit different. With Processing, you control drawing more in an imperative way. Processing has been around for a quite a while, and plays an increasingly important role in the &lt;a href=&quot;http://en.wikipedia.org/wiki/Maker_culture&quot;&gt;DIY or maker&lt;/a&gt; culture.&lt;/p&gt;

&lt;p&gt;To understand the background of Processing, let’s look at this code:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var line = [0, 0, 40, 0];

function sketchProc(processing) {
  processing.setup = function() {
    console.log(&quot;* setup *&quot;);
  }

  processing.draw = function() {
    processing.translate(20, 30);
    processing.line(line[0], line[1], line[2], line[3]);
  }
}
var canvas = document.getElementById(&quot;canvas1&quot;);
var processingInstance = new Processing(canvas, sketchProc);
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;With Processing, you differentiate between 2 stages of drawing: The setup (processing.setup) and actual drawing (processing.draw). Since the “draw” stage is looped over, you can easily build frames for animations. Basic geometrical processing happens directly with coordinates.&lt;/p&gt;

&lt;h2 id=&quot;conclusion&quot;&gt;Conclusion&lt;/h2&gt;

&lt;p&gt;D3 and Processing look great to build info graphics. With Processing, you can easily build advanced interfaces for drawing shapes and interactions, while D3’s toolbox comes with many layout helpers out of the box. Working with graphics in a browser is still young, but it certainly will have a great future.&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>AQL for Twitter Analysis</title>
   <link href="https://thinkingonthinking.com/aql-for-twitter-analysis/"/>
   <updated>2014-05-21T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/aql-for-twitter-analysis</id>
   <content type="html">&lt;p&gt;The Twitter API is great to tap into machine readable representations of Twitter data. In this post, we explore a small combination of a Node script together with ArangoDB.&lt;/p&gt;

&lt;p&gt;As ArangoDB is built around the V8 JavaScript engine, it is possible to load and run JavaScript modules from the ArangoDB console to explore data. This post discusses preliminary ideas on connecting to Twitter and basic AQL functions.&lt;/p&gt;

&lt;h2 id=&quot;authentication&quot;&gt;Authentication&lt;/h2&gt;

&lt;p&gt;Before you can play with data from Twitter, you must setup authentication for API calls to Twitter. This is the first hurdle to access Twitter data. To make requests to the Twitter API, you must signup for a free Twitter developer account and register your app.&lt;/p&gt;

&lt;p&gt;For this, you can go to &lt;a href=&quot;https://apps.twitter.com/app/new&quot;&gt;dev.twitter.com/app/new&lt;/a&gt; and signup your app. After successful signup, you’ll get an API key and an API secret. These token are needed to verify that an request is coming from your app.&lt;/p&gt;

&lt;p&gt;Next, you will find some URLs:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;App-only authentication: https://api.twitter.com/oauth2/token
Request token URL: https://api.twitter.com/oauth/request_token
Authorize URL: https://api.twitter.com/oauth/authorize
Access token URL: https://api.twitter.com/oauth/access_token
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;With these URLs, your app can request access tokens for a user. This might be interesting if you want to analyze data for multiple Twitter users. In order to authorize an application for a user, you can use this script &lt;a href=&quot;https://gist.github.com/mulderp/22174782ebe36bce0c50&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;lookup-twitter-users&quot;&gt;Lookup Twitter Users&lt;/h2&gt;

&lt;p&gt;Once your Twitter user has an access token, you can make all the request to the Twitter API that are documented &lt;a href=&quot;https://dev.twitter.com/docs/api/1.1&quot;&gt;at dev.twitter.com&lt;/a&gt;. For our data analysis experiments, things are getting interesting from &lt;a href=&quot;https://dev.twitter.com/docs/api/1.1#94&quot;&gt;Friends &amp;amp; Followers&lt;/a&gt; section.&lt;/p&gt;

&lt;p&gt;Let’s first fetch our friends. Since the OAuth requests can get a bit hairy, Twitter provides an OAuth tool, see the following screenshot:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/twitter_oauth.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;For example, if you go to the documentation of &lt;a href=&quot;https://dev.twitter.com/docs/api/1.1/get/users/lookup&quot;&gt;GET /users/lookup&lt;/a&gt;, you’ll see a button “Generator OAuth Signature”. Clicking through the different options, you eventually end up with an example curl request, that might look similar to:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ curl --get &apos;https://api.twitter.com/1.1/users/lookup.json&apos; --data &apos;screen_name=twitterapi%2Ctwitter&apos; --header &apos;Authorization: OAuth oauth_consumer_key=&quot;....&quot;, oauth_nonce=&quot;....&quot;, oauth_signature=&quot;....&quot;, oauth_signature_method=&quot;HMAC-SHA1&quot;, oauth_timestamp=&quot;1400617420&quot;, oauth_token=&quot;.....&quot;, oauth_version=&quot;1.0&quot;&apos; --verbose
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;If you run this curl request from the command line, you should see something like:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;[{&quot;id&quot;:6253282,&quot;id_str&quot;:&quot;6253282&quot;,&quot;name&quot;:&quot;Twitter API&quot;,&quot;screen_name&quot;:&quot;twitterapi&quot;,&quot;location&quot;:&quot;San Francisco, CA&quot;,&quot;description&quot;:&quot;The Real Twitter API. ...&quot;}]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Let’s replace the manual curl call with a small Node script:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var request = require(&apos;request&apos;);
var qs = require(&apos;querystring&apos;);

var CONSUMER_KEY = &apos;....&apos;;
var CONSUMER_SECRET = &apos;...&apos;;
var access_token = &apos;....&apos;;
var access_token_secret = &apos;....&apos;;

var oauth = { consumer_key: CONSUMER_KEY,
  consumer_secret: CONSUMER_SECRET,
  token: access_token,
  token_secret: access_token_secret
};

var friends = &apos;https://api.twitter.com/1.1/friends/ids.json&apos;;
var lookup = &apos;https://api.twitter.com/1.1/users/lookup.json?&apos;;

request.get({url: friends, oauth: oauth, json: true}, function(e, r, resp) {

  if (e) {
    console.log(&apos;error: &apos;, e);
    return;
  }

  console.log(resp.ids.length);
  lookup += &apos;user_id=&apos; + resp.ids.slice(1,100);
  console.log(lookup);

  request.get({url: lookup, oauth: oauth, json: true}, function (e, r, users) {
    if (e) {
      console.log(&apos;error: &apos;, e);
      return
    }
    console.log(users.length);
    fs.writeFile(&apos;friends.json&apos;, JSON.stringify(users));
  });
})
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;If all goes well, you’ll have a 100 friends in a file friends.json&lt;/p&gt;

&lt;h2 id=&quot;aql-basics&quot;&gt;AQL Basics&lt;/h2&gt;

&lt;p&gt;Let’s do some data manipulation with AQL and ArangoDB next. First, you must import your Twitter data into the database.&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ arangoimp --create-collection true --collection mulpat_friends friends.json

----------------------------------------
database:         _system
collection:       mulpat_friends
create:           yes
file:             friends.json
type:             json
connect timeout:  3
request timeout:  300
----------------------------------------
Starting JSON import...
2014-05-21T05:33:43Z [76325] INFO processed 32768 bytes (11.42%) of input file
2014-05-21T05:33:43Z [76325] INFO processed 65536 bytes (22.83%) of input file
2014-05-21T05:33:43Z [76325] INFO processed 98304 bytes (34.25%) of input file
2014-05-21T05:33:43Z [76325] INFO processed 131072 bytes (45.66%) of input file
2014-05-21T05:33:43Z [76325] INFO processed 163840 bytes (57.08%) of input file
2014-05-21T05:33:43Z [76325] INFO processed 196608 bytes (68.49%) of input file
2014-05-21T05:33:43Z [76325] INFO processed 229376 bytes (79.91%) of input file
2014-05-21T05:33:43Z [76325] INFO processed 262144 bytes (91.32%) of input file
2014-05-21T05:33:43Z [76325] INFO processed 287050 bytes (100.00%) of input file

created:          99
errors:           0
total:            99
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Let’s explore the data with AQL. To run AQL, I prefer going to your local ArangoDB instance at http://0.0.0.0.8529 and select AQL. &lt;a href=&quot;http://0.0.0.0:8529/_db/_system/_admin/aardvark/standalone.html#query&quot;&gt;This&lt;/a&gt; should be a shortcut. Your AQL console should look similar to:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/aql_editor.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;For the first query, we want to get a quick overview on the friends at a Twitter location:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;for u in mulpat_friends
  limit 2
  filter u.location == &apos;San Francisco&apos;
  return u.screen_name
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The syntax should be self-explaining, but for reference, you might want to checkout the &lt;a href=&quot;https://www.arangodb.org/manuals/current/Aql.html&quot;&gt;AQL docs&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;The query above returns:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;[
  &quot;Fullstack node.js developer&quot;,
  &quot;Labs Manager @WeWork&quot;
]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Often it is nice to aggregate information. With AQL, the main statement is ‘collect’ and its syntax is as:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;for u in mulpat_friends
  collect city = u.location into g
  return {city: city}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This results into:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;[
  {
    &quot;city&quot;: &quot;Amsterdam&quot;
  },
  {
    &quot;city&quot;: &quot;Barcelona&quot;
  },
  {
    &quot;city&quot;: &quot;Barcelona, the World&quot;
  },
  {
    &quot;city&quot;: &quot;bavaria and berlin&quot;
  },
  {
    &quot;city&quot;: &quot;Bavaria/Berlin &quot;
  }
  // ...
]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;You can see already that grouping friends to a location might be difficult since it would require parsing. But with this “imperfect” data, what is the location where we have most friends? This question can be answered by sorting to the length of members in a group:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;for u in mulpat_friends
  collect city = u.location into g
  sort length(g) desc
  return {city: city, count: length(g)}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This results into:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;[
  {
    &quot;city&quot;: &quot;&quot;,
    &quot;count&quot;: 13
  },
  {
    &quot;city&quot;: &quot;San Francisco, CA&quot;,
    &quot;count&quot;: 5
  },
  {
    &quot;city&quot;: &quot;Munich, Germany&quot;,
    &quot;count&quot;: 4
  },
  {
    &quot;city&quot;: &quot;Munich&quot;,
    &quot;count&quot;: 4
  },
  ...    ]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Last, AQL also allows subqueries. For example, in what locations could we reach most followers? To answer this question, you could the following nested query:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;for u in mulpat_friends
  collect city = u.location into g
  sort length(g) desc
  return { 
           city: city, count: length(g),
           friends_count: (for f in g return f.u.friends_count)
         }
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This query would return something like:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;  {
    &quot;city&quot;: &quot;San Francisco, CA&quot;,
    &quot;count&quot;: 5,
    &quot;friends_count&quot;: [
      112,
      573,
      1007,
      49,
      286
    ]
  },
  {
    &quot;city&quot;: &quot;Munich, Germany&quot;,
    &quot;count&quot;: 4,
    &quot;friends_count&quot;: [
      995,
      38,
      48,
      246
    ]
  },
  {
    &quot;city&quot;: &quot;Munich&quot;,
    &quot;count&quot;: 4,
    &quot;friends_count&quot;: [
      25,
      282,
      47,
      63
    ]
  },
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Or, you could ask, in what locations live most of Twitter influencers, based on calculating a ratio between friends and followers:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;for u in mulpat_friends
  collect city = u.location into g
  sort length(g) desc
  return {city: city, count: length(g), friends_count: (for f in g return f.u.friends_count/f.u.followers_count)}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Of course this would need to be averaged out.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Layouts with Backbone</title>
   <link href="https://thinkingonthinking.com/layouts-with-backbone/"/>
   <updated>2014-05-20T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/layouts-with-backbone</id>
   <content type="html">&lt;p&gt;Arranging multiple view templates on the client side is not so easy. Some server-side environments, e.g. Ruby-on-Rails application, support the concept of “application layout” to define how multiple view templates (or “partials”) are arranged.&lt;/p&gt;

&lt;p&gt;How to apply this idea of a layout on the client-side web application with e.g. Backbone?&lt;/p&gt;

&lt;p&gt;Since Backbone is a low-level library, there are multiple approaches how to solve this on higher abstraction levels.&lt;/p&gt;

&lt;h1 id=&quot;attach-views-to-existing-dom-nodes&quot;&gt;Attach Views to existing DOM nodes&lt;/h1&gt;

&lt;p&gt;A simple approach to arrange multiple views in a Backbone application is by delegation. A nice explanation on how this works is given by &lt;a href=&quot;http://twitter.com/ianstormtaylor&quot;&gt;@ianstormtaylor&lt;/a&gt; &lt;a href=&quot;http://ianstormtaylor.com/break-apart-your-backbonejs-render-methods/&quot;&gt;in this post on how to break apart Backbone’s render method&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;The idea is to provide a “layout” from a basic HTML that is generated on first page load or from a parent view:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;lt;div id=&quot;main&quot;&amp;gt;
  &amp;lt;aside id=&quot;sibebar&quot;&amp;gt;
  &amp;lt;/aside&amp;gt;
  
  &amp;lt;section id=&quot;main&quot;&amp;gt;
  &amp;lt;/section&amp;gt;
&amp;lt;/div&amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;On these DOM nodes, Backbone views can attached by using the &lt;a href=&quot;http://backbonejs.org/#View-setElement&quot;&gt;setElement&lt;/a&gt; function of Backbone views. While this approach nicely delegates events from some basic HTML layout, dynamically “swapping” views, e.g. the main view, still would require some manual setups. You would need to take care to unbind existing views and event handlers from the DOM element in order to avoid zombie views.&lt;/p&gt;

&lt;h1 id=&quot;using-thorax-or-handlebones&quot;&gt;Using Thorax or Handlebones&lt;/h1&gt;

&lt;p&gt;A nice strategy to easily build a layout for views, and easily swap DOM elements as needed, is given by the &lt;a href=&quot;http://thoraxjs.org/api.html#thorax.layout-view&quot;&gt;Thorax.LayoutView&lt;/a&gt; idea. Here, you place a Handlebars helper in a template with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;{ { layout-element } }
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And now, whenever you call from a view:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;layout.setView(someView);
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The old view is automatically cleaned up.&lt;/p&gt;

&lt;p&gt;The idea works for placing sidebars, headers and footers too. You can place helpers in your layout with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;{ {view header} }
{ {view sidebar} }
{ {view footer} }
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And, in your views, you can set the child views with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;function initialize() {
  this.header = new Header();
  this.sidebar = new Sidebar();
  this.footer = new Footer();
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The Handlebars “view” helper lookups a child view, will render this and insert it into the DOM as needed. Also, the view helper takes care of unbinding and cleaning up event handler when views are swapped.&lt;/p&gt;

&lt;p&gt;More on the view helpers in Thorax can be found &lt;a href=&quot;http://thoraxjs.org/api.html#template-helpers&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;handlebones&quot;&gt;Handlebones&lt;/h2&gt;

&lt;p&gt;Recently, Ryan Eastridge of &lt;a href=&quot;http://formidablelabs.com/&quot;&gt;FormidableLabs&lt;/a&gt; published a minimalistic version for combining Backbone and Handlebars. The result is &lt;a href=&quot;https://github.com/FormidableLabs/handlebones#helpers&quot;&gt;Handlebones&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;With this, you can re-attach event listeners on subviews, after the parent view triggers the “ready” event. Arranging multiple views with Handlebones and attaching sub-view rendering to the “ready” event, is basically a variation of the “setDelegate” idea from the beginning.&lt;/p&gt;

&lt;p&gt;For example, if you have a LayoutView, you could listen for the “ready” event as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var Layout = Handlebones.View.extend({

  template: layoutTemplate,

  _renderSubviews: function() {
    this.$el.html(this.template());

    var productsDiv = this.$el.find( &quot;ul#products&quot; );
    this.productsList.setElement(productsDiv);
    this.productsList.trigger(&apos;render&apos;);

    // add more subviews here as needed
  },

  initialize: function(options) {
    this.productsList = new ProductList({
      selectable: true,
      collection: options.products,
      modelView: ProductView
    });

    // this triggers rendering the subviews, when the parent view is ready
    this.on(&apos;ready&apos;, _.bind(this._renderSubviews, this));
});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Of course, there are many more approaches how to work with parent and childviews, and how to arrange multiple views with Backbone.&lt;/p&gt;

&lt;p&gt;You might want to search for “subview” at &lt;a href=&quot;http://backplug.io&quot;&gt;http://backplug.io&lt;/a&gt; or directly at &lt;a href=&quot;http://npmjs.org&quot;&gt;http://npmjs.org&lt;/a&gt;.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>The Value of Iteration</title>
   <link href="https://thinkingonthinking.com/the-value-of-iteration/"/>
   <updated>2014-05-14T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/the-value-of-iteration</id>
   <content type="html">&lt;p&gt;Telling a story for an audience is not easy. Especially, if your story includes elements of live coding or some abstractions that might not resonate with the background of an audience.&lt;/p&gt;

&lt;p&gt;This is why preparation of a talk matters. Preparing a story gives you the opportunity to apply learnings from previous talks or experiments. While preparing a talk, you have the opportunity to recover from failures and to provide missing pieces that make sense as a whole. While preparing, you can put together a better story “design”.&lt;/p&gt;

&lt;p&gt;As often with design, it is hard to say, what design exactly means. What I can say is that good design is something that “sticks”. With design, something feels better and fits a context such, that boundaries disappear.&lt;/p&gt;

&lt;p&gt;As most design involve some visuals, design is increasingly an important part of a talk. Over the span of a talk, you deal with a number of slide-ware problems. You must deal with the design of single slides, and, maybe more difficult, you must design a “story” of slides.&lt;/p&gt;

&lt;p&gt;Yet, as with any question of design, answers are hardly resulting from analytic thought. Answers result from facing emotions of an audience or from your inner gut feeling. Answers also result from accepting successes and failures from previous versions of a design.&lt;/p&gt;

&lt;p&gt;The initial version of a design does not need to come from you though. This is where the quote of Picasso might come in, that says: &lt;a href=&quot;https://www.youtube.com/watch?v=CW0DUg63lqU&quot;&gt;“Good artists copy, great artists steal.”&lt;/a&gt; If some design stuck with you, it might stick with others.&lt;/p&gt;

&lt;p&gt;So, find out what sticks with you. What sticks with you and why? What was a reader, user or audience taking away from a previous design? What design might stick with you or a friend next year? Now, design a story from that, and rinse and repeat.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>A Yeoman generator for Backbone projects</title>
   <link href="https://thinkingonthinking.com/a-yeoman-generator-for-backbone-and-browserify/"/>
   <updated>2014-05-02T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/a-yeoman-generator-for-backbone-and-browserify</id>
   <content type="html">&lt;p&gt;By using a code generator, you can speed up the productivity for writing software. Asking the question whether you should use code generator, is a bit similar to choosing between &lt;a href=&quot;http://www.manualversusautomatic.com/&quot;&gt;manual or automatic transmission&lt;/a&gt; when selecting a car. Automatic transmission (and code generators) can give less interruptions to the flow of driving. However, shifting gears manually (or typing code manually) gets easier with practice, and can ensure that you consume the minimum amount of energy.&lt;/p&gt;

&lt;h2 id=&quot;yeoman&quot;&gt;Yeoman&lt;/h2&gt;

&lt;p&gt;Assuming you have a need for project automation, let’s have a look &lt;a href=&quot;http://yeoman.io/&quot;&gt;Yeoman&lt;/a&gt;. Yeoman was inspired by the scaffolding idea in &lt;a href=&quot;http://en.wikipedia.org/wiki/Scaffold_(programming)&quot;&gt;Ruby-on-Rails&lt;/a&gt;. In contrast to Ruby-on-Rails, Yeoman can support all kinds of automation for projects. The important abstraction that enables this flexibility are &lt;a href=&quot;http://yeoman.io/generators.html&quot;&gt;Yeoman-generators&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;I met Yeoman first via the fantastic &lt;a href=&quot;https://github.com/walmartlabs/generator-thorax&quot;&gt;Yeoman generator for Thorax&lt;/a&gt;, but other very interesting Yeoman generators are the &lt;a href=&quot;https://github.com/yeoman/generator-ember&quot;&gt;Ember&lt;/a&gt; and &lt;a href=&quot;https://github.com/yeoman/generator-backbone&quot;&gt;Backbone&lt;/a&gt; generator.&lt;/p&gt;

&lt;p&gt;To understand what a generator does, it makes sense to inspect the “/app/templates” directory. For example, in the &lt;a href=&quot;https://github.com/yeoman/generator-ember/tree/master/app/templates&quot;&gt;templates directory&lt;/a&gt; of the Ember generator you can see templates for a Gruntfile, Bower and Karma.&lt;/p&gt;

&lt;p&gt;Actually, most generators that I saw support an automated setup with Grunt or &lt;a href=&quot;http://thinkingonthinking.com/intro-to-gulp/&quot;&gt;Gulp&lt;/a&gt;, and often use RequireJS. For sandboxing a Node application, I currently prefer CommonJS and Browserify however. This gives nice combination with a &lt;a href=&quot;http://thinkingonthinking.com/scripting-a-csv-converter/&quot;&gt;REPL&lt;/a&gt; on the server and browser.&lt;/p&gt;

&lt;h2 id=&quot;a-generator-for-browserify-and-handlebars&quot;&gt;A generator for Browserify and Handlebars&lt;/h2&gt;

&lt;p&gt;Since I found no Yeoman generator supporting this etup, it became time to write a Yeoman generator. As a start you can use the Yeoman generator-generator, and scaffold some basic application template. Then, you can easily setup a package.json where you include the required Node modules. Right now, I skip Gulp or Grunt, and just include a basic Makefile with a command to call browserify.&lt;/p&gt;

&lt;p&gt;To optimally use Browserify, it can be handy to create symlinks into a directory node_modules, e.g.:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ mkdir -p app/views
$ mkdir app/node_modules
$ cd app/node_modules
$ ln -sf ../views
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;With this symlink, you can require any module easily as require(‘views/foo’).&lt;/p&gt;

&lt;p&gt;Yet, this is open, as I haven’t found out how to create symlinks with Yeoman.&lt;/p&gt;

&lt;p&gt;The current result is the Yeoman &lt;a href=&quot;https://www.npmjs.org/package/generator-backbone-browserify&quot;&gt;generator-backbone-browserify&lt;/a&gt; and its source is at &lt;a href=&quot;https://github.com/mulderp/generator-backbone-browserify&quot;&gt;Github&lt;/a&gt;&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>TDD or BDD? Try RDD!</title>
   <link href="https://thinkingonthinking.com/scripting-a-csv-converter/"/>
   <updated>2014-04-30T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/scripting-a-csv-converter</id>
   <content type="html">&lt;p&gt;NOTE: This article is not about RDD in the sense of Rage Driven Development.&lt;/p&gt;

&lt;p&gt;Kent Beck recently summarized the advantages of &lt;a href=&quot;https://www.facebook.com/notes/kent-beck/rip-tdd/750840194948847&quot;&gt;TDD&lt;/a&gt; in response to David Heinemeier Hansson’s &lt;a href=&quot;http://david.heinemeierhansson.com/2014/tdd-is-dead-long-live-testing.html&quot;&gt;complains&lt;/a&gt; about &lt;a href=&quot;http://david.heinemeierhansson.com/2014/test-induced-design-damage.html&quot;&gt;TDD/BDD&lt;/a&gt; to design an API. It is old wisdom to look at a third option, when you found two opposites poles. This third option might be: &lt;b&gt;RDD&lt;/b&gt;, and can add some extra ideas to the discussion of what “TDD/BDD” and “testing” might mean.&lt;/p&gt;

&lt;p&gt;&lt;b&gt;RDD stands for REPL Driven Development.&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;And &lt;a href=&quot;http://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop&quot;&gt;REPL&lt;/a&gt; recursively stands for the Read-Eval-Print-Loop. The pioneers of RDD might be computers scientists Gerald Jay Sussman and Hal Abelson, as they mention the REPL loop in &lt;a href=&quot;http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-10.html#%_idx_206&quot;&gt;Ch. 1&lt;/a&gt; of their famous book &lt;a href=&quot;http://mitpress.mit.edu/sicp/full-text/book/book.html&quot;&gt;Structure and Interpretation of Computer Programs&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;First, some principles that drive TDD/BDD and “testing”, such as the &lt;a href=&quot;http://en.wikipedia.org/wiki/KISS_principle&quot;&gt;KISS&lt;/a&gt; and &lt;a href=&quot;http://en.wikipedia.org/wiki/Don%27t_repeat_yourself&quot;&gt;DRY principles&lt;/a&gt;, have strong influences in RDD too. A REPL provides great feedback to obtain a feeling for the boundaries of your modules, objects and messages.  And, if you work with data in JSON, a REPL can support you to explore and improve data structures too.&lt;/p&gt;

&lt;h2 id=&quot;the-problem&quot;&gt;The problem&lt;/h2&gt;

&lt;p&gt;Let’s start with the typical problem for RDD. You got some fresh data from several sources that you must insert into some other application, such as the database of a movie festival. Unfortunately, the movie theatres provided you with movies in CSV format (probably from Excel spreadsheets…) . So, you have these tasks:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Read the raw CSV data&lt;/li&gt;
  &lt;li&gt;Convert the CSV format into JSON&lt;/li&gt;
  &lt;li&gt;Provide a maximum amount of code reuse&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;First, you need some raw data in a file ‘movies.csv’:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;title;year ;director;genres
The Artist  ;2009;  Hazanavicius;Comedy,Drama
Dead Man Down;2013;Niels Arden Oplev;Action, Crime
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;So, this is a header line, and 2 rows of data all separated with “;”.&lt;/p&gt;

&lt;h2 id=&quot;entering-the-repl&quot;&gt;Entering the REPL&lt;/h2&gt;

&lt;p&gt;Now, let’s start with the nucleus of RDD development: You fire up a REPL console. For convenience, this example uses a Node console, but it could be the REPL of Ruby, CoffeeScript, Python, Elixir, …&lt;/p&gt;

&lt;p&gt;So, you type:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ node
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;To load a file from the file system, you need some kind of library that deals with IO. In Node, this is ‘fs’ module, and you can require it as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; fs = require(&apos;fs&apos;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Hmm…. now, how to read that movies.csv file? Is it “fs.load…”, “fs.read..”, “fs.readStream”, … ? Good question. The Node REPL can provide some hints if you type:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; fs
{ Stats: [Function],
  exists: [Function],
  existsSync: [Function],
  readFile: [Function]
  ...
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;In about the 4th line, you see “readFile”. That one looks promising. The Node REPL unfortunately does not have direct support for documentation (but other REPLs have). So, to see how to use “fs.readFile”, you can visit the &lt;a href=&quot;http://nodejs.org/api/fs.html#fs_fs_readfile_filename_options_callback&quot;&gt;docs on the fs module&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;As it turns out, you can call ‘readFile’ as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; fs.readFile(&apos;./movies.csv&apos;, function(e, data) {
... console.log(data);
... })
undefined
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And what the REPL will say is:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; &amp;gt; &amp;lt;Buffer 74 69 74 6c 65 3b ...&amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Ok, this looks like data. You would have expected some string in a human readable form, wouldn’t you? The REPL says you got a Buffer. OK, keep calm.&lt;/p&gt;

&lt;p&gt;You can easily run the same command mutliple times in a good REPL.&lt;/p&gt;

&lt;p&gt;So, you try this:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; &amp;gt; fs.readFile(&apos;./movies.csv&apos;, function(e, data) {
 ... console.log(data.toString());
 ... })
 &amp;gt; title;year ;director;genres
   The Artist   ;2009;  Hazanavicius;Comedy,Drama
   Dead Man Down;2013;Niels Arden Oplev;Action, Crime
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Nice! That looks like something you can work with. And, we can mark our first Todo as “done”.&lt;/p&gt;

&lt;h2 id=&quot;save-your-learnings&quot;&gt;Save your learnings&lt;/h2&gt;

&lt;p&gt;In this first REPL session, you were lucky to adapt the same command with minor variations. But our converter will get more complex in a second. So, to keep your mind sane, it’s good to leave the REPL, and put your first abstraction into a file:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ cat &amp;gt; jsony.js
var fs = require(&apos;fs&apos;);
var JSONy = function() {};
module.exports = JSONy
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;With this, you created a JSONy module based on an empty constructor function (think of “class” in other languages).&lt;/p&gt;

&lt;p&gt;Then, you take that ‘fs.readFile’ idea, and add a ‘load’ function to ‘jsony.js’:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;JSONy.load = function(filename) {
  fs.readFile(filename, function(e, data) {
    var row = data.toString();
    console.log(row);
  });
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This is a good start to enter our next REPL session.&lt;/p&gt;

&lt;h2 id=&quot;repling-out-the-convert-function&quot;&gt;REPLing out the convert function&lt;/h2&gt;

&lt;p&gt;To check that the learnings from the first REPL session are in place, let’s start another REPL session:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ node
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Then:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; jsony = require(&apos;./jsony&apos;);
{ [Function] load: [Function] }
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Cool, the load function is already there to play with in the Node console. Let’s try this:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; jsony.load(&apos;./movies.csv&apos;)
undefined
&amp;gt; title;year ;director;genres
The Artist   ;2009;  Hazanavicius;Comedy,Drama
Dead Man Down;2013;Niels Arden Oplev;Action, Crime
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Fantastic, this was not rocket science, but taking small steps is good practice in RDD too.&lt;/p&gt;

&lt;p&gt;To come back to our original goal, converting CSV into JSON, you need several steps. Your load function misses something important: A hook where data is transformed. As Node is non-blocking, you could use a callback, a transform stream or a promise chain to exploit some forms of concurrency.&lt;/p&gt;

&lt;p&gt;Although streams and Promises are great, let’s learn about an important REPL principle: [KISS (Keep it Stupid Simple)(http://en.wikipedia.org/wiki/KISS_principle). For us, it means callbacks. You can add a callback hook in the load function of ‘jsony.js’ as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var fs = require(&apos;fs&apos;);
var JSONy = function() {};

JSONy.load = function(filename, cb) {
  fs.readFile(filename, cb);
}
module.exports = JSONy
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;With a fresh REPL session, you are now ready to explore the main convert function. Let’s start with the following idea for ‘convert’:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; jsony = require(&apos;./jsony&apos;);
&amp;gt; convert = function(e, raw) { return raw.toString().split(&apos;\n&apos;) }
&amp;gt; jsony.load(&apos;./movies.csv&apos;, convert);
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Nothing seems to happens. But in the REPL, you can easily check if this is “no news is good news”, or something was piped into “/dev/null”.&lt;/p&gt;

&lt;p&gt;To check, let’s introduce an array to store the output:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; out = [];
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And, adapt the convert function. Aha, hmm… maybe the “return” above is not required? Important feedback.&lt;/p&gt;

&lt;p&gt;With an output variable, you can simple do:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; convert = function(e, raw) { out = raw.toString().split(&apos;\n&apos;); } 
&amp;gt; jsony.load(&apos;./movies.csv&apos;, convert);
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;When you try this, and inspect ‘out’:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; out
[ &apos;title;year ;director;genres&apos;,
  &apos;The Artist   ;2009;  Hazanavicius;Comedy,Drama&apos;,
  &apos;Dead Man Down;2013;Niels Arden Oplev;Action, Crime&apos;,
  &apos;&apos;, ]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Next, you could wrap your new function:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; out = [];
&amp;gt; _extractRows = function(raw) { return raw.toString().split(&apos;\n&apos;) }
&amp;gt; convert = function(e, raw) { out = _extractRows(raw) }
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;REPLing this up, gives:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; jsony.load(&apos;./movies.csv&apos;, convert);
&amp;gt; out
[ &apos;title;year ;director;genres&apos;,
  &apos;The Artist   ;2009;  Hazanavicius;Comedy,Drama&apos;,
  &apos;Dead Man Down;2013;Niels Arden Oplev;Action, Crime&apos;,
  &apos;&apos; ]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;OK, again, some important learnings.The previous insights and new sketches for code ideas are the right amount of code to add into the JSONy module as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;function _extractRows(raw) {
  return raw.toString().split(&apos;\n&apos;)
}

function _splitFields(line) {
  return line.split(&apos;;&apos;);
}

var out = [];
function convert(e, raw, cb) {
  var lines = _extractRows(raw);

  for (var i=0; i &amp;lt; lines.length; i++) {
    var line = lines[i];
    var fields = _splitFields(line);
    out.push(fields);
  } 

  cb(e, out);
}

jsony.convert = convert;

module.exports = jsony
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Note the unoffical convention to mark a “private” function in JavaScript: With a ‘_’ prefix, you can inform readers of your code that this is not available outside the JSONy module. Also, we add a callback to the ‘convert’ function, such that ‘convert’ can be more easily be used with IO operations.&lt;/p&gt;

&lt;h2 id=&quot;keeping-it-dry&quot;&gt;Keeping it DRY&lt;/h2&gt;

&lt;p&gt;As your module is beginning to take shape, you can start to use the command line as REPL too. This will keep you from typing the same things over and over, and it’s here where you meet the &lt;a href=&quot;http://en.wikipedia.org/wiki/Don%27t_repeat_yourself&quot;&gt;DRY (don’t repeat yourself) principle&lt;/a&gt;) of RDD.&lt;/p&gt;

&lt;p&gt;In a new file ‘convert.js’, you add a shell for the JSONy module:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ cat &amp;gt; convert.js
var jsony = require(&apos;./jsony&apos;);

jsony.load(&apos;./movies.csv&apos;, function(e, raw) {
  jsony.convert(e, raw, function(e, result) {
    console.log(result);
  });
});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now, from the command line you type:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ node convert.js
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And, you again get the data:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;[ [ &apos;title&apos;, &apos;year &apos;, &apos;director&apos;, &apos;genres&apos; ],
  [ &apos;The Artist   &apos;, &apos;2009&apos;, &apos;  Hazanavicius&apos;, &apos;Comedy,Drama&apos; ],
  [ &apos;Dead Man Down&apos;, &apos;2013&apos;, &apos;Niels Arden Oplev&apos;, &apos;Action, Crime&apos; ],
  [ &apos;&apos; ] ]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;the-last-mile&quot;&gt;The last mile&lt;/h2&gt;

&lt;p&gt;Turning an array into JSON is not difficult, is it? It is even easier by applying a bit of functional magic with e.g. &lt;a href=&quot;http://underscorejs.org&quot;&gt;Underscore&lt;/a&gt;. For example, you can add in ‘jsony.js’:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var head = _splitFields(lines[0]);
out.push(_.object(head, fields));
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now, you can push almost JSON objects to your screen:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ node convert.js 
[ { title: &apos;title&apos;,
    &apos;year &apos;: &apos;year &apos;,
    director: &apos;director&apos;,
    genres: &apos;genres&apos; },
  { title: &apos;The Artist   &apos;,
    &apos;year &apos;: &apos;2009&apos;,
    director: &apos;  Hazanavicius&apos;,
    genres: &apos;Comedy,Drama&apos; },
  { title: &apos;Dead Man Down&apos;,
    &apos;year &apos;: &apos;2013&apos;,
    director: &apos;Niels Arden Oplev&apos;,
    genres: &apos;Action, Crime&apos; } ]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The missing step is:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; JSON.stringify(out);
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Before trying this, let’s cleanup some abstractions. It is a good idea to remove trailing whitespaces, and add an exception if there is something wrong with the header. You’ll end up with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;JSONy.load = function(filename, cb) {
  fs.readFile(filename, cb);
}

function _extractRows(raw) {
  return raw.toString().split(&apos;\n&apos;)
}

function _trimArray(array) {
  return array.map(function(str) { return str.trim() });
}

function _splitFields(line) {
  return _trimArray(line.split(&apos;;&apos;));
}

var out = [];
function convert(e, raw, cb) {

  if (!raw) {
    cb(&quot;no input&quot;, null);
    return
  }

  var lines = _extractRows(raw);

  var head = JSONy.head;

  // use first line as header if available
  if (JSONy.HEAD) {
    start = JSONy.HEAD;
    head = _splitFields(lines[0]);
  }

  if (!head) {
    cb(&quot;No header specified&quot;, null);
    return;
  }

  for (var i=0; i &amp;lt; lines.length-1; i++) {
    var line = lines[i];
    var fields = _splitFields(line);
    out.push(_.object(head, fields));
  } 

  cb(e, out);
}

JSONy.HEAD = 1; // use first line as header
JSONy.convert = convert;

module.exports = JSONy
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And, your command line client:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;#!/usr/bin/env node

var jsony = require(&apos;./jsony&apos;);

jsony.load(process.argv[2], function(e, raw) {
  jsony.convert(e, raw, function(e, result) {
    if (e) {
      console.log(e);
      process.exit(1);
    }
    console.log(JSON.stringify(result));
  });
});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This is runnable when you do:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ chmod u+x ./convert.js
$ ./convert.js movies.csv
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And you should get:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;[{&quot;title&quot;:&quot;title&quot;,&quot;year&quot;:&quot;year&quot;,&quot;director&quot;:&quot;director&quot;,&quot;genres&quot;:&quot;genres&quot;},{&quot;title&quot;:&quot;The Artist&quot;,&quot;year&quot;:&quot;2009&quot;,&quot;director&quot;:&quot;Hazanavicius&quot;,&quot;genres&quot;:&quot;Comedy,Drama&quot;},{&quot;title&quot;:&quot;Dead Man Down&quot;,&quot;year&quot;:&quot;2013&quot;,&quot;director&quot;:&quot;Niels Arden Oplev&quot;,&quot;genres&quot;:&quot;Action, Crime&quot;}]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h1 id=&quot;conclusions&quot;&gt;Conclusions&lt;/h1&gt;

&lt;p&gt;You saw how to apply the KISS and DRY coding principles with RDD. RDD stands for REPL Driven Development, and REPL recursively for Read-Print-Eval-Loop.&lt;/p&gt;

&lt;p&gt;RDD can be practioned in every dynamic environment that supports a REPL, such as JavaScript, CoffeeScript, Python, Ruby, Elixir or Bash.&lt;/p&gt;

&lt;p&gt;The main idea of RDD is to get feedback by running code in a REPL. Also with RDD, it is important to stay calm while coding. That means you shouldn’t fall into the trap of Rage Driven Development or other options. RDD might not work for every one or every company, but it does for me.&lt;/p&gt;

&lt;p&gt;You can have a look at the &lt;a href=&quot;https://github.com/mulderp/jsony&quot;&gt;github repo&lt;/a&gt; or at &lt;a href=&quot;https://www.npmjs.org/package/jsony&quot;&gt;jsony@npmjs.org&lt;/a&gt; - Note, that JavaScript was not taken by coincidence here to illustrate RDD. Most of the ideas of this post sum up ideas from a talk I gave at the &lt;a href=&quot;http://2014.nosql-matters.org/cgn/&quot;&gt;NoSQLmatters conference&lt;/a&gt; about running JavaScript in a database with ArangoDB. You can find the slides &lt;a href=&quot;https://speakerdeck.com/mulderp/script-your-way-to-nosql&quot;&gt;here&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Thanks for reading, let me know how to imrprove the ideas presented here!&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>An Intro to Gulp</title>
   <link href="https://thinkingonthinking.com/intro-to-gulp/"/>
   <updated>2014-04-12T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/intro-to-gulp</id>
   <content type="html">&lt;p&gt;The JavaScript community is increasingly moving towards &lt;a href=&quot;http://en.wikipedia.org/wiki/Stream_(computing)&quot;&gt;Streams&lt;/a&gt; to read, transform and write large chunks of data. One area, where multiple data transformations are needed is in frontend build processes.&lt;/p&gt;

&lt;p&gt;Especially for larger projects, when you add linting, transforms and an uglify step, it can make sense to automate builds.&lt;/p&gt;

&lt;p&gt;But let’s start with a small project: Automate the build of a simple browserify command.&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ browserify ./app/main.js &amp;gt; static/bundle.js
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;To run this with gulp with uglify support, you need to setup a gulpfile.js.&lt;/p&gt;

&lt;p&gt;But first, you need to install gulp:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ npm install -g gulp
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And, in your project folder you should add a local gulp runner too:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ npm install gulp --save-dev
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Next, you need to install some gulp tasks:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ npm install vinyl-source-stream --save-dev
$ npm install gulp-streamify --save-dev
$ npm install gulp-uglify --save-dev
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The gulp tasks are the main steps of a build, and you define those in a gulpfile.js:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var browserify = require(&apos;browserify&apos;);
var gulp = require(&apos;gulp&apos;);
var source = require(&apos;vinyl-source-stream&apos;);
var jstify = require(&apos;jstify&apos;);
var streamify = require(&apos;gulp-streamify&apos;);
var uglify = require(&apos;gulp-uglify&apos;);

gulp.task(&apos;browserify&apos;, function() {
  return browserify(&apos;./app/main.js&apos;)
    .transform(&apos;jstify&apos;)
    .bundle()
    .pipe(source(&apos;bundle.js&apos;))
    .pipe(streamify(uglify()))
    .pipe(gulp.dest(&apos;build/&apos;));
});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;What happens here is: You first setup browserify to use a ‘jstify’ transform and bundle the input file ‘app/main.js’. The resulting stream is piped into ‘bundle.js’, which again is piped into the uglify transform, and saved under ‘build’.&lt;/p&gt;

&lt;p&gt;This task is called ‘browserify’ and you can call it from the command as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ gulp browserify
[gulp] Using gulpfile /.../gulptest/gulpfile.js
[gulp] Starting &apos;browserify&apos;...
[gulp] Finished &apos;browserify&apos; after 6.58 s
pmu:gulp(master|76m) pmu$ ls -ltr build/
total 384
-rw-r--r--  1 pmu  staff  194062 12 Apr 17:51 bundle.js
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;references&quot;&gt;References&lt;/h2&gt;

&lt;ul&gt;
  &lt;li&gt;https://www.npmjs.org/package/vinyl-source-stream&lt;/li&gt;
  &lt;li&gt;http://markgoodyear.com/2014/01/getting-started-with-gulp/&lt;/li&gt;
  &lt;li&gt;http://www.codefellows.org/blogs/quick-intro-to-gulp-js&lt;/li&gt;
  &lt;li&gt;Thanks to geson in #browserify for pointing me in the right directions&lt;/li&gt;
&lt;/ul&gt;
</content>
 </entry>
 
 <entry>
   <title>Tracking Time</title>
   <link href="https://thinkingonthinking.com/tracking-time/"/>
   <updated>2014-04-11T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/tracking-time</id>
   <content type="html">&lt;p&gt;One of the big differences between working as employee and working as freelancer is: The experience of time.&lt;/p&gt;

&lt;p&gt;When I was employed, time evaporated. You try to get at work before 9AM (not always making it), being productive for one hour, having the daily standup around 10h30 (this &lt;a href=&quot;http://vooza.com/videos/standup-meetings/&quot;&gt;Vooza persiflage&lt;/a&gt; is just great), being bored until noon, lunch, finally finding flow for some time in the afternoon, lasting - in the best case - until 6PM.&lt;/p&gt;

&lt;p&gt;Finally, heading home and working on side-projects that are fun.&lt;/p&gt;

&lt;p&gt;As a freelancer, time feels totally different. When a day starts, you must write emails to get new projects. Then, you empathize with potential clients to understand their problems. You try to squeeze in some time for programming and writing. During lunch, you feel like being in a networking event. Back from lunch, the flow experience starts immediatly since there all these great options and new combinations of interests that match both, yours and those of your clients.&lt;/p&gt;

&lt;p&gt;The music as freelancer feels much more inspiring. May be, because you are more creative. Switching off all distractions (like Twitter, Hacker news, newspapers, skype) feels much more natural. In the evenings, the trade-off of networking events and private time and important time-outs is much more important.&lt;/p&gt;

&lt;p&gt;Then, after some days, the days arrive that you count the hours that you worked.&lt;/p&gt;

&lt;p&gt;Personal accounting of a freelancer is a really, really big difference from being employed. As freelancer, hours directly map to income. Yet, these hours are different from the 8 hours of a normal working day.&lt;/p&gt;

&lt;p&gt;When you are employed, you account hours for one single project, day in and day out. As freelancer, you must account time to multiple projects a day. It is much more important to have a plan for a day, or at least for a couple of hours to get some work done. You always have the base worries of an empty project pipeline (= no income) and growing your reputation (= delivering high quality work).&lt;/p&gt;

&lt;p&gt;As freelancer, output is more important than presence. As employee, presence can be more important.&lt;/p&gt;

&lt;p&gt;Working as freelancer for sure is riskier. Being employed, can yield a higher income. Yet helping people and businesses when they have real pains, or being part of greenfield projects, makes the time as freelancer very rewarding so far. And, in contrast to being employed, counting hours can help you plan and set priorities, instead of someone else’s.&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Software and customer's jobs</title>
   <link href="https://thinkingonthinking.com/customers-jobs/"/>
   <updated>2014-04-05T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/customers-jobs</id>
   <content type="html">&lt;p&gt;To examine needs for consulting services, a discussion with &lt;a href=&quot;http://www.grahamclarkphoto.com/&quot;&gt;Graham Clark from San Francisco&lt;/a&gt; reminded me to look at the &lt;a href=&quot;http://www.businessmodelgeneration.com/downloads/value_proposition_canvas.pdf&quot;&gt;Value Proposition Canvas&lt;/a&gt; from Alexander Osterwalder. When designing services and products, Osterwalder suggests to take a look 3 segments in a circle: Customer jobs, pains and gains.&lt;/p&gt;

&lt;p&gt;In this post, I want to explore a bit the first question about “customer jobs”, see the right part in the figure below.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/value_proposition_canvas.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;When a company buys consulting hours for services on software development, what are the goals and/or jobs of a customer?&lt;/p&gt;

&lt;p&gt;On a first thought companies hire consultants for these reasons:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;Avoid research costs: Know-how mostly originates from research, but research is cost intensive. So, a customer might try to find people and technologies to reduce research costs. This might be why many software and engineering consultancies are ending with “lab”, e.g. like &lt;a href=&quot;http://pivotallabs.com/&quot;&gt;Pivotal-labs&lt;/a&gt;, &lt;a href=&quot;http://envylabs.com/&quot;&gt;Envy-Labs&lt;/a&gt;, &lt;a href=&quot;http://formidablelabs.com/&quot;&gt;Formidable-Labs&lt;/a&gt;, &lt;a href=&quot;http://www.metalabdesign.com/&quot;&gt;Metalab&lt;/a&gt; or inhouse research like &lt;a href=&quot;http://www.bell-labs.com/history/unix/&quot;&gt;Bell-labs&lt;/a&gt; or &lt;a href=&quot;http://en.wikipedia.org/wiki/Philips_Natuurkundig_Laboratorium&quot;&gt;Nat-lab&lt;/a&gt;&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Solve “one-time” problems: There are situations where software is needed only “one-time”, e.g. to process data from an unique CRM campaign, to process a portfolio acquisition or a certain kind of measurement, … Companies just want to solve that custom problem, without investing into much research or building a team around that one-time problem. One-time problems also give nice online services, e.g. &lt;a href=&quot;http://slicecraft.com/&quot;&gt;Slicecraft&lt;/a&gt; or other PSD-2-Sass conversion services.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Improve “social status”: For certain kinds of jobs, it might “sound” better if it is done by an external consultancy. Management consultancies might work a bit in that area. If you look at McKinsey, BCG or other top management consultancies, they offer some kind of social status around their services. In software, this might be the reason that companies hire freelancers working with products from Microsoft or Google. Placing yourself close to these companies, attracts a somewhat social status too.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Apart from the jobs of companies for freelancers, it might be interesting to look at the pains companies have with their current software solutions or freelancers. This might be the topic of another post.&lt;/p&gt;

&lt;p&gt;Let me know what you think of your customer’s jobs.&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>One week freelance</title>
   <link href="https://thinkingonthinking.com/first-week-as-freelancer/"/>
   <updated>2014-04-04T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/first-week-as-freelancer</id>
   <content type="html">&lt;p&gt;When you start as a freelancer, you need contacts, or a so-called “network” to build upon. You need to email potential customers, if they have time for a chat, if they work with freelancers and whether your background might match with the person they are looking for.&lt;/p&gt;

&lt;p&gt;After my first week of freelance, I made a small business model fiddle about what my situation looks like right now.&lt;/p&gt;

&lt;p&gt;&lt;img style=&quot;max-width:1200px&quot; src=&quot;/static/images/bm_fiddle_freelance.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;There is a lot more going on than programming only. In fact, most things feel like marketing and selling. This is not bad though, since in the spirit of &lt;a href=&quot;http://www.kalzumeus.com/start-here-if-youre-new/&quot;&gt;Patrick McKenzie&lt;/a&gt;, selling is an important part of a software business, and for me at least, for making software products.  McKenzie seem to talk about the &lt;a href=&quot;http://vimeo.com/81693597&quot;&gt;software business&lt;/a&gt; here too.&lt;/p&gt;

&lt;p&gt;So for week 1, a key acitivity was building and filling up my project pipeline. To do this, there is only one way: Talk with people. A good idea might also be of giving talks at meetups and conferences, and have people talk with you. After living some years in Munich, this strategy looks best to fill up the initial project pipeline. And I am lucky to work with some great freelancers in Munich from the start.&lt;/p&gt;

&lt;p&gt;A second important activity of acquiring projects is writing emails. Writing emails is more difficult though, since you need to have email addresses of leads, you need to send those emails (in the form of email campaigns) and you need some time to figure out what to write.&lt;/p&gt;

&lt;p&gt;A good use of an email channel sounds like important way to communicate with customers on the longer term though. Also, sending emails will be important when you will start selling products, instead of hours. So, I played with some ideas &lt;a href=&quot;http://thinkingonthinking.com/build-your-email-list-with-firebase/&quot;&gt;to capture email addresses with a landingpage&lt;/a&gt;. It looks it is possible to create a small email list manager, and integrate some APIs for cheap delivery.&lt;/p&gt;

&lt;p&gt;Then there is the question of how to work with software recruiters. These people live from their email lists, and they could help you filling a project pipeline. I am not yet too sure what to think about recruiters. They have their own interest in placing you in projects, that might not be the best for you. Those kinds of projects, where you replace permanent staff, instead of providing real consulting hours on specific problems. But the spectrum of recruiters looks broad enough to give it a try for some weeks.&lt;/p&gt;

&lt;p&gt;Last, the first questions on accounting and taxes popped up. There are a number of tools such as &lt;a href=&quot;http://getquaderno.com/&quot;&gt;Quaderno&lt;/a&gt;, &lt;a href=&quot;http://moneybird.com&quot;&gt;Moneybird&lt;/a&gt; and other that help you with invoices. So far, I seem to have a good accountant who can help me understanding the language of the tax office.&lt;/p&gt;

&lt;p&gt;So, what are your experiences on selling consulting hours? What marketing do you do?&lt;/p&gt;

&lt;h2 id=&quot;inspirations&quot;&gt;Inspirations&lt;/h2&gt;

&lt;ul&gt;
  &lt;li&gt;Part of this post was inspired by discussions with &lt;a href=&quot;http://www.grahamclarkphoto.com/3-essential-marketing-tactics-photographers/&quot;&gt;Graham Clark&lt;/a&gt;. As creatives, photographers have to solve similar problems, as software developers need to do.&lt;/li&gt;
&lt;/ul&gt;
</content>
 </entry>
 
 <entry>
   <title>Your Email List with Firebase and Backbone</title>
   <link href="https://thinkingonthinking.com/build-your-email-list-with-firebase/"/>
   <updated>2014-03-31T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/build-your-email-list-with-firebase</id>
   <content type="html">&lt;p&gt;Forms in landing pages are great for capturing interest in a product or service. Quite easily, you can start collecting email addresses with a static page, by using a bit of JavaScript and a “Backend-as-a-Service”.&lt;/p&gt;

&lt;p&gt;The idea of &lt;a href=&quot;http://en.wikipedia.org/wiki/Static_web_page&quot;&gt;a static web page&lt;/a&gt; is serving HTML with JavaScript without using server-side page updates. This can be interesting when you build a dashboard or a landing page, as we will discuss here. The advantage of building apps as static pages, is that you can avoid the costs for maintaining infrastructure that come with dynamic web pages.&lt;/p&gt;

&lt;p&gt;But where do you store data from forms in static pages? Or, how could you display new data to a user when a page is “static”? To answer these questions, we can look at the idea of combining JavaScript with a &lt;a href=&quot;http://nobackend.org/&quot;&gt;noBackend&lt;/a&gt;, or so-called “Backend-as-a-Service”. The name “no-Backend” is certainly inspired by “no-SQL”, pointing towards application design with flexible schema’s and applying different mindsets for writing and reading data. Besides agile development, approaches for “Backend-as-a-Service” support you with keeping maintenance and infrastructure operations at a minimum.&lt;/p&gt;

&lt;p&gt;For the impatient reader, you can take a quick look at &lt;a href=&quot;http://thinkingonthinking.com/landingpage&quot;&gt;the landingpage demo&lt;/a&gt; to see whether a static page with a noBackend can make sense for you.&lt;/p&gt;

&lt;h2 id=&quot;firebase&quot;&gt;Firebase&lt;/h2&gt;

&lt;p&gt;There are a number of “noBackend” providers offering simple API hosting. One of the most advanced is &lt;a href=&quot;https://www.firebase.com/&quot;&gt;Firebase&lt;/a&gt;. With Firebase’s &lt;a href=&quot;https://www.firebase.com/pricing.html&quot;&gt;Hacker Plan&lt;/a&gt;, you can store up to 100 MB of data for free, which should be enough for collecting emails with a landing page.&lt;/p&gt;

&lt;p&gt;To get started with Firebase, you can signup with your Github account or simply with your email. On Firebase’s dashboard, you then can create as many apps as you like.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/firebase_dashboard.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;So, with the dashboard, let’s create a store for emails. I’ll call this “mulpat”, since it is my Twitter handle. The idea is to store emails in a collection “contacts”. Collections in Firebase can be populated either with a data import, or, with some JavaScript that is run from the browser console. The latter can be nice to quickly reset or setup some predefined data. Let’s look at how it is done.&lt;/p&gt;

&lt;p&gt;First, we point a Firebase reference to a collection, e.g.&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var ref = new Firebase(&quot;https://mulpat.firebaseio.com/contacts&quot;);
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;We now can “set” data or “push” data into the reference with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;ref.set({id: &quot;patrick&quot;, email: &quot;mulder@mtpartners.de&quot;});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This should look like the following:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/firebase_set.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Be careful with “set” though. It will wipe out existing data from the collection.&lt;/p&gt;

&lt;p&gt;Once collections are created, you can also apply rules for security. &lt;a href=&quot;https://www.youtube.com/watch?v=IGlzbmnAlRQ&quot;&gt;This video&lt;/a&gt; is well worth watching.&lt;/p&gt;

&lt;p&gt;To prevent others from reading our contacts list, we can setup security and validation rules in Firebase. A very basic approach could like the following:&lt;/p&gt;

&lt;p&gt;{
       “rules”: {
           “.read”: false,
           “.write”: false,
           “contacts”: {
             “$other”: {
               “.write”: true,
               “email”: {
                 “.validate”: “newData.isString()”
               }
             }
           }
       }
   }&lt;/p&gt;

&lt;p&gt;So, with these basics, we can setup a static page and connecting a form to the Firebase bucket.&lt;/p&gt;

&lt;h2 id=&quot;backbone-setup&quot;&gt;Backbone Setup&lt;/h2&gt;

&lt;p&gt;For the landing page setup, we want to keep the stack for the static page as simple as possible. This means, we want to avoid too many abstractions from workflow automation. Therefore, the build tool of choice is &lt;a href=&quot;http://thinkingonthinking.com/unix-in-the-browser/&quot;&gt;Browserify&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;First, we setup some directories and link the node_modules directory. By symlinking node_modules, we can easily require the libs from there. We should have a directory layout similar to:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;|-app
|---collections
|---node_modules
|---views
|-lib
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;In node_modules, there are symlinks to collections, views and libs.&lt;/p&gt;

&lt;p&gt;We also can get some node_modules with npm, such as jQuery, Underscore and Backbone. For this, we do:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ npm init
$ npm install jquery-untouched --save-dev
$ npm install backbone --save-dev
$ npm install underscore --save-dev
$ npm install client-backfire --save-dev
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This last line is the adapter to synch a collection with Firebase. We come back to this in a second.&lt;/p&gt;

&lt;p&gt;We also need some static HTML in index.html:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; &amp;lt;!DOCTYPE html&amp;gt;
 &amp;lt;html lang=&quot;en&quot;&amp;gt;
 &amp;lt;head&amp;gt;
 	&amp;lt;meta charset=&quot;utf-8&quot;&amp;gt;
 	&amp;lt;title&amp;gt;Landing Page&amp;lt;/title&amp;gt;
 	&amp;lt;meta name=&quot;description&quot; content=&quot;&quot;&amp;gt;
     &amp;lt;link href=&quot;http://fonts.googleapis.com/css?family=Open+Sans:300,600,700&quot; rel=&quot;stylesheet&quot; &amp;gt;
 	&amp;lt;meta name=&quot;viewport&quot; content=&quot;width=device-width, initial-scale=1.0&quot;&amp;gt;
     &amp;lt;link rel=&quot;stylesheet&quot; href=&quot;./static/style.css&quot; /&amp;gt;
     &amp;lt;script src=&quot;./static/bundle.js&quot;&amp;gt;&amp;lt;/script&amp;gt;
     &amp;lt;script&amp;gt;
       var app = require(&quot;app&quot;);
     &amp;lt;/script&amp;gt;
 &amp;lt;/head&amp;gt;
 &amp;lt;body&amp;gt;
 
 &amp;lt;div class=&quot;container-main&quot;&amp;gt;
     &amp;lt;div class=&quot;header&quot;&amp;gt;
         &amp;lt;h1&amp;gt;Welcome to a Landingpage!&amp;lt;/h1&amp;gt;
         &amp;lt;h3&amp;gt;If you like what you see, please leave your email!&amp;lt;/h3&amp;gt;
     &amp;lt;/div&amp;gt;
 
     &amp;lt;div class=&quot;form-wrapper&quot;&amp;gt;
       &amp;lt;form id=&quot;contact&quot;&amp;gt;
         &amp;lt;input type=&quot;text&quot; name=&quot;email&quot; /&amp;gt;
         &amp;lt;input type=&quot;submit&quot; value=&quot;SEND&quot; class=&quot;button-blue&quot; &amp;gt;
       &amp;lt;/form&amp;gt;
     &amp;lt;/div&amp;gt;
 &amp;lt;/div&amp;gt;
 
 &amp;lt;/body&amp;gt;
 &amp;lt;/html&amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This is just a basic form, with an input element for emails, and a “send” button. We can also add &lt;a href=&quot;http://codepen.io/rexkirby/pen/Fdnlz&quot;&gt;some style&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Next, we create a simple Backbone setup. First, we create a simple collection that references the contacts collection at Firebase. The module client-backfire wraps the Backfire adapter in a CommonJS setup. Backfire provides a Sync adapter for Firebase.&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var backfire = require(&quot;client-backfire&quot;);
var Backbone = backfire.Backbone;

var Contacts = Backbone.Firebase.Collection.extend({
  firebase: &quot;https://mulpat.firebaseio.com/contacts&quot;
});

var contacts = new Contacts();
module.exports = contacts
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;After this basic data layer, we write a view that listens to the form submit event. As we bind the contact collection upon view instantiation, we can simply reference the collection, and render a small “thank you” page. Putting these ideas together, the view looks like:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var Backbone = require(&quot;backbone&quot;);
var $ = Backbone.$;

var ContactView = Backbone.View.extend({
  events: {
    &quot;submit&quot;: &quot;addContact&quot;
  },
  addContact: function(ev) {
    ev.preventDefault();
    var name = $(&quot;input[name=&apos;email&apos;]&quot;);
    this.collection.add({email: name.val()});
    this.renderThanks();
  },
  renderThanks: function() {
    this.$el.html(&quot;thanks&quot;);
  },
  initialize: function() {
    this.listenTo(this.collection, &quot;all&quot;, function(ev) {
      console.log(ev);
    });
  }
});

module.exports = ContactView;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;To help our understanding of Firebase, we can add temporarily a monitor for events, with the “listenTo … all” syntax in the view constructor.&lt;/p&gt;

&lt;p&gt;Last, we bind the model to the view, map the dependencies and finish the application setup with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var $ = require(&quot;jquery-untouched&quot;);
var Backbone = require(&quot;backbone&quot;);
Backbone.$ = $;

var contacts = require(&quot;collections/contacts&quot;);
var ContactForm = require(&quot;views/contact&quot;);

var view;
$(document).ready(function() {
  view = new ContactForm({el: $(&quot;#contact&quot;), collection: contacts });
});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now we can bundle the setup with Browserify:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ browserify -r ./app/main.js:app &amp;gt; static/bundle.js
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;You can see the complete &lt;a href=&quot;https://github.com/mulderp/landingpage&quot;&gt;Github repository here&lt;/a&gt;. By using a branch “gh-pages”, we also obtain easily a static website and here is the &lt;a href=&quot;http://thinkingonthinking.com/landingpage/&quot;&gt;live demo&lt;/a&gt;&lt;/p&gt;

&lt;h2 id=&quot;conclusions&quot;&gt;Conclusions&lt;/h2&gt;

&lt;p&gt;The application could still be improved in a number of ways. We could wire up events from Firebase with Zapier and Mailgun for example, to automatically send emails when data is submitted. The view still misses a number of things such as validation of an email address, and maybe some more helpers to build a social community around your landing page. Keep tuned, updates will be coming.&lt;/p&gt;

&lt;h2 id=&quot;references&quot;&gt;References&lt;/h2&gt;

&lt;ul&gt;
  &lt;li&gt;A blog on Static Apps: &lt;a href=&quot;http://www.staticapps.org/&quot;&gt;http://www.staticapps.org/&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;More background on API’s: &lt;a href=&quot;https://zapier.com/learn/apis/&quot;&gt;https://zapier.com/learn/apis/&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;Firebase overview on security rules: https://www.firebase.com/docs/security/security-rules.html&lt;/li&gt;
  &lt;li&gt;An advanced example of security rules: https://github.com/firebase/firechat/blob/master/rules.json&lt;/li&gt;
&lt;/ul&gt;
</content>
 </entry>
 
 <entry>
   <title>About Two Way Data Binding with Backbone.Stickit</title>
   <link href="https://thinkingonthinking.com/two-way-databinding-with-stickit/"/>
   <updated>2014-03-23T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/two-way-databinding-with-stickit</id>
   <content type="html">&lt;p&gt;Forms with live preview are a common use case for 2-way databinding. To help you understand whether &lt;a href=&quot;http://pipefishbook.com&quot;&gt;using Backbone.js&lt;/a&gt; can be an option for your app, let’s look at an example of 2-way databinding with Backbone.&lt;/p&gt;

&lt;h2 id=&quot;why-do-we-need-2-way-data-binding&quot;&gt;Why do we need 2-way data binding?&lt;/h2&gt;

&lt;p&gt;With 2-way data binding, your UI can provide short feedback loops on inputs from users. Observing input on forms are common when building browser editors, or advanced control elements, like sliders that are coupled with numeric input.&lt;/p&gt;

&lt;p&gt;In a technical sense, the idea behind 2-way databinding is managing model-view updates in two directions. First, views are updated when a model changes, which is the default case for 1-way databinding. Additionally, views can directly update models, which is a second update “direction”, hence the name “2-way databinding”.&lt;/p&gt;

&lt;h2 id=&quot;a-business-card-editor&quot;&gt;A business card editor&lt;/h2&gt;

&lt;p&gt;To show what this means, let’s look at an example. As a freelancer, the first thing you need is business cards. I found no online business cards provider that offer a simple online business card editor. So, let’s build a basic one. You can see a working demo of the discussion &lt;a href=&quot;http://thinkingonthinking.com/businesscards/&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Backbone.js is a library, not a framework. This means you add other libraries to build out some larger apps. There are a number of data-binding plugins in the Backbone ecosystem, but I chose &lt;a href=&quot;http://nytimes.github.io/backbone.stickit/&quot;&gt;Backbone.Stickit&lt;/a&gt;. This plugin can be nicely fetched with npm and packaged with &lt;a href=&quot;http://thinkingonthinking.com/unix-in-the-browser/&quot;&gt;Browserify&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;So, first, let’s create a project directory, and init the project:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ npm init
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Next, we install the dependencies:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ npm install jquery-untouched --save-dev
$ npm install underscore --save-dev
$ npm install backbone --save-dev
$ npm install backbone.stickit --save-dev
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Then, we setup a directory for the bundle.js from Browserify:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ mkdir static
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Last, we create and link some paths for the views and models:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ mkdir -p app/views
$ mkdir -p app/models
$ mkdir -p app/node_modules
$ cd app/node_modules
$ ln -sf ../views
$ ln -sf ../models
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Our project directory now looks as:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ tree -l
   |-app
   |---collections
   |---models
   |---node_modules
   |---views
   |-node_modules
   |-static
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;In order to have a common jQuery and Backbone dependency in the project, we must point the jQuery dependency to Backbone.
In “app/main.js”, we can do the following:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; // setup deps
 var $ = require(&quot;jquery-untouched&quot;);
 var _ = require(&quot;underscore&quot;);
 var Backbone = require(&quot;backbone&quot;);
 Backbone.$ = $;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now that the dependencies and directories are setup, we can start working on our editor.&lt;/p&gt;

&lt;h2 id=&quot;the-address-model&quot;&gt;The address model&lt;/h2&gt;

&lt;p&gt;One of the core concepts behind Backbone.js is to separate UI controls (events, rendering) from application state (models, collections). So, let’s setup first a small address model, that can be used later in the editor view:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; var Backbone = require(&quot;backbone&quot;);
  
 var Address = Backbone.Model.extend({
   defaults: {
     &quot;name&quot;: &quot;Baloo The Bear&quot;,
     &quot;email&quot;: &quot;balu@junglebook.com&quot;,
     &quot;street&quot;: &quot;Middle of Nowhere 1&quot;,
     &quot;zip&quot;: &quot;12345&quot;,
     &quot;city&quot;: &quot;Amazon Delta&quot;,
     &quot;country&quot;: &quot;Brasil&quot;,
     &quot;phone&quot;: &quot;555-123123&quot;,
     &quot;web&quot;: &quot;https://www.youtube.com/watch?v=lz0J88gnINc&quot;
   }
 });
 module.exports = Address;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This model is just seeting attributes to default values. In the case of 1-way data binding, a Backbone.View would bind to changes of this model. In the case of 2-way data-binding, a view can update a model directly. We come to this in a second.&lt;/p&gt;

&lt;h2 id=&quot;the-views&quot;&gt;The views&lt;/h2&gt;

&lt;p&gt;We basically will have two views: One editor view and one preview pane, where the business card will be rendered. The way the view is constructed is a bit different from a normal Backbone.View as will be clear in a second.&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var Backbone = require(&quot;backbone&quot;);
var Stickit = require(&quot;backbone.stickit&quot;);

var NewAddress = Backbone.View.extend({
   bindings: {
      &apos;#name&apos;: &apos;name&apos;,
      &apos;#email&apos;: &apos;email&apos;,
      &apos;#street&apos;: &apos;street&apos;,
      &apos;#zip&apos;: &apos;zip&apos;,
      &apos;#city&apos;: &apos;city&apos;,
      &apos;#country&apos;: &apos;country&apos;,
      &apos;#phone&apos;: &apos;phone&apos;,
      &apos;#web&apos;: &apos;web&apos;
   },

   render: function() {
     this.$el.html(&apos; \
      &amp;lt;h2&amp;gt;Edit your details&amp;lt;/h2&amp;gt; \
      &amp;lt;ul&amp;gt; \
      &amp;lt;li&amp;gt;&amp;lt;label for=&quot;name&quot;&amp;gt;Name&amp;lt;/label&amp;gt;&amp;lt;input id=&quot;name&quot; name=&quot;name&quot; type=&quot;text&quot;&amp;gt;&amp;lt;/li&amp;gt; \
      &amp;lt;li&amp;gt;&amp;lt;label for=&quot;email&quot;&amp;gt;Email&amp;lt;/label&amp;gt;&amp;lt;input id=&quot;email&quot; name=&quot;email&quot; type=&quot;text&quot;&amp;gt;&amp;lt;/li&amp;gt; \
      &amp;lt;li&amp;gt;&amp;lt;label for=&quot;street&quot;&amp;gt;Street&amp;lt;/label&amp;gt;&amp;lt;input id=&quot;street&quot; name=&quot;street&quot; type=&quot;text&quot;&amp;gt;&amp;lt;/li&amp;gt; \
      &amp;lt;li&amp;gt;&amp;lt;label for=&quot;zip&quot;&amp;gt;ZIP&amp;lt;/label&amp;gt;&amp;lt;input id=&quot;zip&quot; name=&quot;zip&quot; type=&quot;text&quot;&amp;gt;&amp;lt;/li&amp;gt; \
      &amp;lt;li&amp;gt;&amp;lt;label for=&quot;city&quot;&amp;gt;City&amp;lt;/label&amp;gt;&amp;lt;input id=&quot;city&quot; name=&quot;city&quot; type=&quot;text&quot;&amp;gt;&amp;lt;/li&amp;gt; \
      &amp;lt;li&amp;gt;&amp;lt;label for=&quot;country&quot;&amp;gt;Country&amp;lt;/label&amp;gt;&amp;lt;input id=&quot;country&quot; name=&quot;country&quot; type=&quot;text&quot;&amp;gt;&amp;lt;/li&amp;gt; \
      &amp;lt;li&amp;gt;&amp;lt;label for=&quot;phone&quot;&amp;gt;Phone&amp;lt;/label&amp;gt;&amp;lt;input id=&quot;phone&quot; name=&quot;phone&quot; type=&quot;text&quot;&amp;gt;&amp;lt;/li&amp;gt; \
      &amp;lt;li&amp;gt;&amp;lt;label for=&quot;phone&quot;&amp;gt;Web&amp;lt;/label&amp;gt;&amp;lt;input id=&quot;web&quot; name=&quot;web&quot; type=&quot;text&quot;&amp;gt;&amp;lt;/li&amp;gt; \
     &amp;lt;/ul&amp;gt;&apos;);
     this.stickit();
     return this;
   }
});
module.exports = NewAddress;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Notice the new view property “bindings”. This mapping is needed for Backbone.Stickit to &lt;em&gt;bind&lt;/em&gt; inputs to model attributes. After the view has rendered, we call “stickit()” to start observing changes in the view and model in two ways.&lt;/p&gt;

&lt;p&gt;Next, we want to render the address changes in the preview pane. Since we have Backbone.Stickit in our stack anyway, we can simply use stickit for one-way data binding too. The idea for the preview pane then becomes:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; var Backbone = require(&quot;backbone&quot;);
 var Stickit = require(&quot;backbone.stickit&quot;);
 
 var Preview = Backbone.View.extend({
   bindings: {
     &apos;.name&apos;: &apos;name&apos;,
     &apos;.email&apos;: &apos;email&apos;,
     &apos;.street&apos;: &apos;street&apos;,
     &apos;.zip&apos;: &apos;zip&apos;,
     &apos;.city&apos;: &apos;city&apos;,
     &apos;.country&apos;: &apos;country&apos;,
     &apos;.phone&apos;: &apos;phone&apos;,
     &apos;.web&apos;: &apos;web&apos;
   },
   render: function() {
     this.$el.html(&apos;&amp;lt;div class=&quot;name&quot;&amp;gt;&amp;lt;/div&amp;gt;\
       &amp;lt;hr /&amp;gt; \
       &amp;lt;div class=&quot;column&quot;&amp;gt; \
         &amp;lt;div class=&quot;email&quot;&amp;gt;&amp;lt;/div&amp;gt; \
         &amp;lt;div class=&quot;street&quot;&amp;gt;&amp;lt;/div&amp;gt; \
         &amp;lt;span class=&quot;zip&quot;&amp;gt;&amp;lt;/span&amp;gt;&amp;lt;span class=&quot;city&quot;&amp;gt;&amp;lt;/span&amp;gt; \
       &amp;lt;/div&amp;gt; \
       &amp;lt;div class=&quot;country&quot;&amp;gt;&amp;lt;/div&amp;gt; \
       &amp;lt;div class=&quot;phone&quot;&amp;gt;&amp;lt;/div&amp;gt; \
       &amp;lt;div class=&quot;web&quot;&amp;gt;&amp;lt;/div&amp;gt; \
    &apos;);
     this.stickit();
     return this;
   }
 });
 module.exports = Preview;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;So, we Backbone.Stickit we can leave out quite some “this.listenTo(this.model, …. )” manual bindings in a constructor of a view.
To round up the example, we can initialize the app in the file app/main.js as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var address;
$(document).ready(function() {
  address = new Address();
  var formView = new FormView({ model: address, el: $(&quot;#newForm&quot;)});
  formView.render();

  var preview = new Preview({model: address, el: $(&quot;#preview&quot;) });
  preview.render();
});


module.exports = {
  address: function() { return address }
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The last module exports is handy to observe the actual address model from the developer console.&lt;/p&gt;

&lt;p&gt;That’s all. For this use case, Backbone has definitively a nice answer to 2-way data binding. What do you think? Any experiences you want to share on binding a UI to state?&lt;/p&gt;

&lt;h2 id=&quot;references&quot;&gt;References&lt;/h2&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;a href=&quot;https://github.com/mulderp/businesscards&quot;&gt;Repository Demo application&lt;/a&gt; and &lt;a href=&quot;http://thinkingonthinking.com/businesscards&quot;&gt;Demo&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;http://www.lucaongaro.eu/blog/2012/12/02/easy-two-way-data-binding-in-javascript/&quot;&gt;VanillaJS data-binding&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;http://nytimes.github.io/backbone.stickit/&quot;&gt;Backbone.Stickit&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://github.com/theironcook/Backbone.ModelBinder&quot;&gt;Backbone.ModelBinder&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;http://docs.angularjs.org/guide/databinding&quot;&gt;Data binding in Angular&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</content>
 </entry>
 
 <entry>
   <title>On Education Innovation</title>
   <link href="https://thinkingonthinking.com/on-education-innovation/"/>
   <updated>2014-03-17T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/on-education-innovation</id>
   <content type="html">&lt;p&gt;There is a well known proverb on the importance of education: “Give a man a fish and you feed him for a day; teach a man how to fish and you feed him for a lifetime.”&lt;/p&gt;

&lt;p&gt;But what does this mean in the times of the internet? What does it mean, when so many concepts and ideas on software are publicly available in the form of open-source software?&lt;/p&gt;

&lt;h1 id=&quot;crowd-education&quot;&gt;“Crowd”-education&lt;/h1&gt;

&lt;p&gt;Open-source software works, because groups of people with similar interests gather to teach and learn together. If we look at popular open-source frameworks, we can always see a mix of public criticism of features, public discovery of bugs and free exchange of information.&lt;/p&gt;

&lt;p&gt;But also, less experienced developers often get valuable input from more experienced developers, by discussions in public chat rooms, on email lists or Q &amp;amp; A forums, such as Stackoverflow.&lt;/p&gt;

&lt;p&gt;Having discoveries made by communities, and finding a common voice, is not only the strength of open-soure software, but also of general web movements, such as crowd-“sourcing”, crowd-“funding”, and “open” innovation in general. I think, we are just at the dawn of a time, where education will be driven by open communities too. For example, some impressive experiments for teaching programming can be seen in &lt;a href=&quot;http://www.youtube.com/watch?v=aiFOurKwy7M&quot;&gt;Pamela Fox’s Keynote at the Fluent Conference&lt;/a&gt;&lt;/p&gt;

&lt;h1 id=&quot;story-telling&quot;&gt;Story-telling&lt;/h1&gt;

&lt;p&gt;We love good stories. How important it is to tell good stories can be seen from cultural studies. The two extremes might be the Bible on one side, and science-fiction on the other. Those “stories” teach us a lot about goals we should have.&lt;/p&gt;

&lt;p&gt;The idea of stories in education might be related to the concept of “cognitive map” that prof. &lt;a href=&quot;https://web.media.mit.edu/~minsky/OLPC-1.html&quot;&gt;Marvin Minsky mentions in the context of the One-Laptop-per-Child project&lt;/a&gt;. Although knowledge is more and more freely available in the form of Wikipedia and Youtube, finding our “personal story” is far from easy.&lt;/p&gt;

&lt;p&gt;Therefore, it is important to teach elements of a good story and how to identify those for your personal one. My guess is that good stories are the result of good questions, and &lt;a href=&quot;http://thinkingonthinking.com/Push-Conference/&quot;&gt;we should practice asking questions, and see all kinds of variations in your personal “life” questions&lt;/a&gt;.&lt;/p&gt;

&lt;h1 id=&quot;knowledge-clusters&quot;&gt;Knowledge clusters&lt;/h1&gt;

&lt;p&gt;To actually make progress in digital education, I believe we need close contact to universities and businesses that are offline. Our digital culture is a continuation of memes that were already at work for centuries (or even thousands of years).&lt;/p&gt;

&lt;p&gt;We learn from each other in situations of direct interactions. It is one thing, to criticise people in an online forum under a pseudonym, but another to actually look someone in the eyes, and praise or criticize someone in the public.&lt;/p&gt;

&lt;p&gt;Therefore, so called “knowledge clusters” are and will be very valuable. These clusters often emerge at the boundaries of universities and companies,  and offer a chance to discuss ideas with experts in 1-1 interactions.&lt;/p&gt;

&lt;p&gt;The Silicon Valley is a great example of having research funds spinning off knowledge and businesses from having generations of smart people teaching each other. Smaller examples can be found in &lt;a href=&quot;http://meetup.com&quot;&gt;local meetups e.g. for programming&lt;/a&gt;, or other kinds of gatherings for learning and sharing.&lt;/p&gt;

&lt;p&gt;Education will change for sure in the next centuries, and the challenge is now to teach right values and respect towards discoveries coming from online and offline worlds.&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Browserify - Unix in the browser</title>
   <link href="https://thinkingonthinking.com/unix-in-the-browser/"/>
   <updated>2014-03-16T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/unix-in-the-browser</id>
   <content type="html">&lt;p&gt;The command-line is an important tool for both, Unix and Node. The command-line supports running and exploring programs. Its philosophy reminds us to think about keeping ideas simple. And, it allows us to compose big ideas from many smaller ones. (The command-line can even be a medium for interactive presentations, as &lt;a href=&quot;http://vimeo.com/62988591&quot;&gt;@substack shows here&lt;/a&gt; )&lt;/p&gt;

&lt;p&gt;One of the most important command-line tools in Node is: &lt;a href=&quot;http://en.wikipedia.org/wiki/Npm_(software)&quot;&gt;npm&lt;/a&gt;. Currently, &lt;a href=&quot;https://www.npmjs.org/&quot;&gt;npmjs&lt;/a&gt;, serves more than 60k JavaScript modules. And, you can access most of these codebases easily from the browser with &lt;a href=&quot;http://browserify.org/&quot;&gt;browserify&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;browserify&quot;&gt;Browserify&lt;/h2&gt;

&lt;p&gt;With browserify, you can run the same code in the browser, that you would run in Node on Unix. This is great, since it allows you to “require” modules from Node, and play with ideas, before you enter the world of the document-object-model (DOM), or the browser.&lt;/p&gt;

&lt;p&gt;Let’s first look at the basics. In Node.js, you could write a “hello world” as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ mkdir lib
$ cat &amp;gt; lib/hello.js
  console.log(&quot;Hello world.&quot;)
$ node lib/hello.js
  Hello world.
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;In order to browsify this server-side script for the browser environment, let’s write:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ browserify ./lib/hello.js
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And we should see something like:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;(function e(t,n,r){function s(o,u){if(!n[o]){if(!t[o]){var a=typeof require==&quot;function&quot;&amp;amp;&amp;amp;require;if(!u&amp;amp;&amp;amp;a)return a(o,!0);if(i)return i(o,!0);throw new Error(&quot;Cannot find module &apos;&quot;+o+&quot;&apos;&quot;)}var f=n[o]={exports:{}};t[o][0].call(f.exports,function(e){var n=t[o][1][e];return s(n?n:e)},f,f.exports,e,t,n,r)}return n[o].exports}var i=typeof require==&quot;function&quot;&amp;amp;&amp;amp;require;for(var o=0;o&amp;lt;r.length;o++)s(r[o]);return s})({1:[function(require,module,exports){
console.log(&quot;Hello world.&quot;)

},{}]},{},[1])
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;What you see above is some wrapper, that enables “require” on the hello module in the browser. It might look a bit heavy for this simple use case, but it will be more fun in a second.&lt;/p&gt;

&lt;h3 id=&quot;require-modules&quot;&gt;Require modules&lt;/h3&gt;

&lt;p&gt;Say, we are working on an application, where we need to clean up and process user inputs. We might want to look at &lt;a href=&quot;https://github.com/epeli/underscore.string#readme&quot;&gt;Underscore.String&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;To install and explore the module on the server, we would write:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ npm install underscore.string
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;We then write a simple module that takes a string and makes it a slug. In &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;lib/slugger.js&lt;/code&gt;, we enter:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var _ = require(&apos;underscore.string&apos;);
module.exports = function(str) {
  return _.slugify(str);
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;To test on the server, we can run Node:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ node
&amp;gt; require(&quot;./lib/slugger&quot;)(&quot;to be or not to be&quot;);
to-be-or-not-to-be
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This should be not much of a surprise either. On a sidenote, the “./” marks a local module. If we would left out this, “require” would walk the “node_modules” path. A bit of background can be found &lt;a href=&quot;https://github.com/maxogden/art-of-node/#how-require-works&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Now, let’s come to the interesting part. We can package this code for the browser, with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ mkdir static
$ browserify -r ./lib/slugger:slugger &amp;gt; static/bundle.js
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The “-r” flag tells browserify to provide a “require” function for the browser. With the “:” we map the module’s path, to an actual module name. To understand why, let’s look at the code in the browser.&lt;/p&gt;

&lt;p&gt;We load the file bundle.js in a browser with basic HTML under static/index.html :&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;lt;html&amp;gt;
&amp;lt;head&amp;gt;
&amp;lt;script src=&quot;bundle.js&quot;&amp;gt;&amp;lt;/script&amp;gt;
&amp;lt;/head&amp;gt;
&amp;lt;body&amp;gt;
&amp;lt;/body&amp;gt;
&amp;lt;/html&amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;If we open that HTML in a developer console, we will have a require keyword now, and we can do:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; var slugger = require(&quot;slugger&quot;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And, we can work again with this module, as we were on the server. Besides having a tool for composing applications from modules, we also can run and test code on the server, without rescueing to browser tests approach such as Jasmine. Last, we can build web applications that partially render templates on the server, while the same logic can be executed in the browser, for partial DOM updates. This approach is actually used in the &lt;a href=&quot;https://github.com/rendrjs/rendr&quot;&gt;rendr&lt;/a&gt; library by Airbnb.&lt;/p&gt;

&lt;h2 id=&quot;resources&quot;&gt;Resources&lt;/h2&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;a href=&quot;http://vimeo.com/62988591&quot;&gt;@substack talk&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;http://learnjs.io/blog/2013/11/24/browserify-resources/&quot;&gt;http://learnjs.io/blog/2013/11/24/browserify-resources/&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</content>
 </entry>
 
 <entry>
   <title>Synching DOM and State</title>
   <link href="https://thinkingonthinking.com/key-value-pairs/"/>
   <updated>2014-03-14T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/key-value-pairs</id>
   <content type="html">&lt;p&gt;Data is often a kind of snapshot of state. While we often represent different aspects of state, we often target for data abstractions that hold in different situations.&lt;/p&gt;

&lt;p&gt;Key-value pairs, or property list, have shown very flexible and clean representation of state. For example, if we want to track a list of products in a shop, we might have a key-value pair similar to:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;{  
   name: &quot;Peligrino&quot;,
   price: 1.5,
   description: &quot;Bottle of water from Italy&quot;
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;In most cases, we track more than one item, or list of pairs. We want to track collections, or arrays, of key-value pairs that might look like:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;[
   { name: &quot;Peligrino&quot;, price: 1.5 },
   { name: &quot;Vittel&quot;, price: 1. 5}
]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now, whereas abstract data structures help you tracking state and computations, you display something else to users. When you are working with web browsers, the resulting problem is the translation of state changes to the document-object-model, or DOM.&lt;/p&gt;

&lt;p&gt;With the DOM, we have a representation for rendering pixels in a web browser, as well as medium to capture events from input devices. Whereas the key-value pairs can apply to a number of different situation, the DOM is rather specific. For example, we might want to render the Peligrino data as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; &amp;lt;h2&amp;gt;Peligrino&amp;lt;/h2&amp;gt;
 &amp;lt;ul&amp;gt;
   &amp;lt;li&amp;gt;&amp;lt;span&amp;gt;Price:&amp;lt;/span&amp;gt; 1.5 Eur&amp;lt;/li&amp;gt;
   &amp;lt;li&amp;gt;Bottle of water&amp;lt;/li&amp;gt;
 &amp;lt;/ul&amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Interestingly, when we translate the state of Peligrino water to DOM nodes, the number of data structures increases. In this case, we add h2, ul, li and span tags, besides the raw data. So, we go from 3 pairs, to 9 DOM nodes. In general, it is more costly to maintain state in the DOM, than it would be in key-value pairs. And, often we have many more items in a collection, than we actually can render to the screen.&lt;/p&gt;

&lt;p&gt;So, tracking state in key-value pairs, outside the DOM, is often a good idea. To translate those pairs to DOM nodes, we can use JavaScript templates, where we replace placeholders with actual data. An example of templating with Underscore might look like this:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; &amp;lt;h2&amp;gt;&amp;lt;%= name %&amp;gt;&amp;lt;/h2&amp;gt;
 &amp;lt;ul&amp;gt;
   &amp;lt;li&amp;gt;&amp;lt;span&amp;gt;Price:&amp;lt;/span&amp;gt; &amp;lt;%= price %&amp;gt; Eur&amp;lt;/li&amp;gt;
   &amp;lt;li&amp;gt;&amp;lt;%= description %&amp;gt;&amp;lt;/li&amp;gt;
 &amp;lt;/ul&amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;When you render the properties into a template and put the result into the DOM, the memory usage increases. This might be a problem if your site becomes larger. Synching state to the DOM must be careful examined for those cases.&lt;/p&gt;

&lt;p&gt;By binding changes in the properties to DOM updates, we also can understand the workings of the MV* pattern, which you can play with &lt;a href=&quot;http://pipefishbook.com/editor&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Observing data changes and updating the DOM can quickly become more complicated. For example, if we had a list:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;[
  { name: &quot;item 1&quot;, votes: 0.5 },
  { name: &quot;item 2&quot;, votes: 0.4 },
  { name: &quot;item 3&quot;, votes: 0.3 }
]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Which we would render into:&lt;/p&gt;

&lt;ul&gt;
    &lt;li&gt;Item 1 &lt;span&gt;50%&lt;/span&gt;&lt;/li&gt;
    &lt;li&gt;Item 2 &lt;span&gt;40%&lt;/span&gt;&lt;/li&gt;
    &lt;li&gt;Item 3 &lt;span&gt;10%&lt;/span&gt;&lt;/li&gt;
  &lt;/ul&gt;

&lt;p&gt;If we would want to sort the data on the votes, while tracking updates on the votes, quite some manual control would be needed. Maybe not for a dozen of data rows, but certainly if the amount of data increases into the hundreds or thousands. It’s here where the Virtual DOM of &lt;a href=&quot;http://facebook.github.io/react/&quot;&gt;React&lt;/a&gt; might help you.&lt;/p&gt;

&lt;p&gt;When dealing with user inputs, observing changes in key-value properties can be tricky. But continuous DOM updates are not advisable either.&lt;/p&gt;

&lt;p&gt;The constraints and needs for synching DOM and state was also the background of &lt;a href=&quot;https://speakerdeck.com/mulderp/backbone-and-handlebars&quot;&gt;a talk that I gave at the JavaScript meetup in San Francisco&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;There are always custom business requirements to consider, when we setup a rendering strategy in a web application. An editor type of application might need better data-binding, than a product browser. A social network with realtime feedback, might need even more advanced rendering strategies. A shop for Ecommerce might support for server-side rendering too.&lt;/p&gt;

&lt;p&gt;Let me know if this post is clear, or if I make some flaws in the argumentation.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Flow and Skills</title>
   <link href="https://thinkingonthinking.com/skill-bars/"/>
   <updated>2014-02-14T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/skill-bars</id>
   <content type="html">&lt;p&gt;One of the most heard question as freelancer is: What are your skills?&lt;/p&gt;

&lt;p&gt;Let’s briefly explore some ideas.&lt;/p&gt;

&lt;p&gt;According to &lt;a href=&quot;http://en.wikipedia.org/wiki/Mihaly_Csikszentmihalyi&quot;&gt;Mihaly Csikszentmihalyi&lt;/a&gt;, the way how we apply our skills to problems defines how we experience work. This is a bit complicated, but the main idea is, if the problems that you tackle match your skills, you experience so-called “flow”. If the challenge levels are much higher than our skills, we experience anxiety, and if we are more skilled than challenged, we feel bored or relaxed.&lt;/p&gt;

&lt;p&gt;So, simply representing skills in bar charts, as is often seen in &lt;a href=&quot;http://dribbble.com/shots/458231-Infographic-Resume&quot;&gt;infographical resumes&lt;/a&gt;, misses something. Therefore, let’s play with an idea how to represent both skills (“what can I do?”) and challenge (“what do I want to do?”).&lt;/p&gt;

&lt;p&gt;We can start with simple skill bars. As &lt;a href=&quot;http://patrick.mtpartners.de&quot;&gt;a start&lt;/a&gt;, these look like:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/skill_bars.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;However, these skill bars are a bit confusing. For example, although I have (almost) finished a book on &lt;a href=&quot;http://thinkingonthinking.com/writing-on-ruby-and-backbone/&quot;&gt;Backbone.js&lt;/a&gt;, my skills in years should be much higher with Ruby-on-Rails (I started around 2009). And it is getting even more difficult to judge on skills, that were applied further in the past (like Java).&lt;/p&gt;

&lt;p&gt;So, to bring some objectivity into this, we might first add a scale just representing years, and no personal judgement. Then, the situation would look like this:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/skill_bars_years.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Yet, something is not yet represented right. I want to get jobs for developing web interfaces with Backbone.js, and rather avoid the Java dolldrums (although much has happened, since my last Java job with JEE 1.4). And although I could get paid for working with Ruby-on-Rails, quite some Rails conventions don’t apply so well for single page applications. So, how could we represent our passions now?&lt;/p&gt;

&lt;p&gt;According to the flow concept, we might try to bring in the challenge part, or, the “I want to” part:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/skill_bars_challenge.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;What do you think about this essay? Is it a viable way to communicate as a freelancer on the market? What might be improved?&lt;/p&gt;

&lt;p&gt;Special thanks to &lt;a href=&quot;http://codepen.io/designil/&quot;&gt;Designil&lt;/a&gt; for pointing at a codepen for skill bars, &lt;a href=&quot;http://www.chris-randall.com/sleepworkers-the-fuel-for-the-factory-corporates/&quot;&gt;Chris Randall&lt;/a&gt; for a post on the flow concept, and &lt;a href=&quot;https://twitter.com/mtrimpe&quot;&gt;Michiel Trimpe&lt;/a&gt; as well as &lt;a href=&quot;https://twitter.com/gausby&quot;&gt;Martin Gausby&lt;/a&gt; for feedback on freelancing.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Getting Started with D3</title>
   <link href="https://thinkingonthinking.com/Getting-Started-With-D3/"/>
   <updated>2014-02-06T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/Getting-Started-With-D3</id>
   <content type="html">&lt;p&gt;Visualizations are great tools to understand ideas and play with data. As Tim Brown from Ideo writes in &lt;a href=&quot;http://designthinking.ideo.com/?p=1294&quot;&gt;this post&lt;/a&gt;&lt;/p&gt;

&lt;blockquote lang=&quot;en&quot;&gt;&lt;p&gt;Visual thinking isn’t limited to illustrations. It can take many forms. Mind maps, two-by-two matrices, and other visual frameworks can help explore and describe ideas in valuable ways that require little more than a few straight lines and some imagination.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;Especially for visualizing data in graphs, &lt;a href=&quot;http://d3js.org/&quot;&gt;D3&lt;/a&gt; might be helpful. D3 stands for “Data-Driven Documents” and is a JavaScript visualization library for HTML and SVG. This post is a short overview to help you get started.&lt;/p&gt;

&lt;h2 id=&quot;loading-d3&quot;&gt;Loading D3&lt;/h2&gt;

&lt;p&gt;The easiest way to load D3 is by loading the script from a CDN, such as:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;lt;script src=&quot;https://cdnjs.cloudflare.com/ajax/libs/d3/3.4.1/d3.min.js&quot;&amp;gt;&amp;lt;/script&amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Alternatively, you can copy the library from the &lt;a href=&quot;https://github.com/mbostock/d3&quot;&gt;github repository&lt;/a&gt; or the &lt;a href=&quot;http://d3js.org&quot;&gt;project website&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Important: By adding a script tag with a reference to the D3.js library, you obtain a “d3” object to work with. With this, you have a tool to render data in a browser.&lt;/p&gt;

&lt;h2 id=&quot;the-d3-object&quot;&gt;The d3 object&lt;/h2&gt;

&lt;p&gt;The d3 object is somewhat similar to the “$” object in jQuery. For example, you can “select” DOM nodes, like a circle tag. And you can build new nodes with “append”.&lt;/p&gt;

&lt;p&gt;The d3 object also helps you with the setup of a convas for rendering graphics.  And, you need a “svg” based canvas. Adding this “svg” is the first step to build a graph. Therefore, let’s define the following construct:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var vis = d3.select(&quot;#graph&quot;)
            .append(&quot;svg&quot;);
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This selects the DOM node with id ‘#graph’. You can then add attributes such as width and height of the graph with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;     var w = 900,
         h = 400;
     vis.attr(&quot;width&quot;, w)
        .attr(&quot;height&quot;, h);
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;You can also add text with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;     vis.text(&quot;The Graph&quot;)
        .select(&quot;#graph&quot;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Selecting and adding DOM nodes should look pretty familiar if you have worked with selectors in jQuery.&lt;/p&gt;

&lt;h2 id=&quot;placing-nodes&quot;&gt;Placing nodes&lt;/h2&gt;

&lt;p&gt;However, D3 differs from jQuery as it allows to “layout” graphical structures according to “data”. This means, &lt;a href=&quot;http://bost.ocks.org/mike/selection/&quot;&gt;a selection of DOM nodes&lt;/a&gt; provides a number of different functions than the same selection with jQuery.&lt;/p&gt;

&lt;p&gt;Once you have a selection of DOM nodes, you can “join” data to the DOM nodes! This is done with the &lt;a href=&quot;https://github.com/mbostock/d3/wiki/Selections#wiki-data&quot;&gt;data()&lt;/a&gt; command. The mapping of data to nodes can feel a bit magical, since this &lt;a href=&quot;http://bost.ocks.org/mike/join/&quot;&gt;adds relationships&lt;/a&gt; based on a declarative syntax. If you end up somewhat confused (as I was), you can read some additional explanation &lt;a href=&quot;http://knowledgestockpile.blogspot.de/2012/01/understanding-selectall-data-enter.html&quot;&gt;here&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;To make a first mapping, we first define nodes:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; var nodes = [{x: 30, y: 50},
              {x: 50, y: 80},
              {x: 90, y: 120}]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Since we start with a fresh canvas, all nodes will be new, and we can map these with “selectAll” - “data” - “enter”:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;  vis.selectAll(&quot;circle .nodes&quot;)
     .data(nodes)
     .enter()
     .append(&quot;svg:circle&quot;)
     .attr(&quot;class&quot;, &quot;nodes&quot;)
     .attr(&quot;cx&quot;, function(d) { return d.x; })
     .attr(&quot;cy&quot;, function(d) { return d.y; })
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The sequence is this: You select a group of DOM nodes (that might exist or not). You then &lt;a href=&quot;http://bost.ocks.org/mike/join/&quot;&gt;join&lt;/a&gt; these nodes with data. For every new DOM node, you specify a tag name (circle) and add a class and more attributes.&lt;/p&gt;

&lt;p&gt;To actually see the circles, you must set the fill attribute:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;  vis.selectAll(&quot;circle .nodes&quot;)
      .data(nodes)
      .enter()
      .append(&quot;svg:circle&quot;)
      .attr(&quot;class&quot;, &quot;nodes&quot;)
      .attr(&quot;cx&quot;, function(d) { return d.x; })
      .attr(&quot;cy&quot;, function(d) { return d.y; })
      .attr(&quot;r&quot;, &quot;10px&quot;)
      .attr(&quot;fill&quot;, &quot;black&quot;) 
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This should result into something like:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/3_circles.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Since the circles are setup, you now can add and remove data, and the graph will update automatically: &lt;a href=&quot;http://codepen.io/mulderp/pen/aDrxq&quot;&gt;Try this codepen&lt;/a&gt;&lt;/p&gt;

&lt;h2 id=&quot;connecting-the-dots&quot;&gt;Connecting the Dots&lt;/h2&gt;

&lt;p&gt;A graph is based on nodes and &lt;em&gt;links&lt;/em&gt;. To render these links, you need lines between the circles. Since you have the coordinates of the circles, you could add lines as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;var links = [
  {source: nodes[0], target: nodes[1]},
  {source: nodes[2], target: nodes[1]}
]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;You can use the &lt;a href=&quot;https://github.com/mbostock/d3/wiki/SVG-Shapes#wiki-svg_line&quot;&gt;line SVG shape&lt;/a&gt; for connecting the dots:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;vis.selectAll(&quot;.line&quot;)
   .data(links)
   .enter()
   .append(&quot;line&quot;)
   .attr(&quot;x1&quot;, function(d) { return d.source.x })
   .attr(&quot;y1&quot;, function(d) { return d.source.y })
   .attr(&quot;x2&quot;, function(d) { return d.target.x })
   .attr(&quot;y2&quot;, function(d) { return d.target.y })
   .style(&quot;stroke&quot;, &quot;rgb(6,120,155)&quot;);
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;We have functions to translate the coordinates to line attributes. The result can be seen &lt;a href=&quot;http://codepen.io/mulderp/pen/wxmBd&quot;&gt;in this codepen&lt;/a&gt;
To see some of the mechanics, you can add and remove data, and see how the graph changes.&lt;/p&gt;

&lt;h2 id=&quot;exploring-further&quot;&gt;Exploring further&lt;/h2&gt;

&lt;p&gt;Setting up graphs manually with nodes and edges might be interesting for small examples. However, if you often need to setup graphs, or the number of nodes and edges increases, a graph can be be setup with an algorithm too. This is where &lt;a href=&quot;http://en.wikipedia.org/wiki/Force-directed_graph_drawing&quot;&gt;force layouts&lt;/a&gt; help.&lt;/p&gt;

&lt;p&gt;With this, you can setup a graph like in &lt;a href=&quot;http://codepen.io/mulderp/full/KGFvx&quot;&gt;this example&lt;/a&gt;:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/graph_levelgraph.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;D3 also supports force layout algorithms, and a nice place to start is &lt;a href=&quot;http://www.d3noob.org/2013/03/what-is-force-layout-diagram-in-d3js.html&quot;&gt;here&lt;/a&gt;. Another option might be using D3 plugins, such as the &lt;a href=&quot;https://github.com/d3/d3-plugins/tree/master/graph&quot;&gt;graph plugin&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;What are your experiences with D3 for graphs and the different approaches? Leave feedback here, or at (hackernews)[https://news.ycombinator.com/item?id=7210162].&lt;/p&gt;

&lt;h2 id=&quot;resources&quot;&gt;Resources&lt;/h2&gt;

&lt;p&gt;If you want to explore further, here are some interesting links:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;&lt;a href=&quot;http://alignedleft.com/tutorials/d3/&quot;&gt;D3 Tutorials by Scott Murray&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;Fine tuning the display of circles: &lt;a href=&quot;http://stackoverflow.com/questions/15859457/d3-js-circles-are-not-appearing&quot;&gt;A question on SO&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;An overview on &lt;a href=&quot;https://www.dashingd3js.com/svg-basic-shapes-and-d3js&quot;&gt;basic shapes&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;Another tutorial &lt;a href=&quot;http://blog.stephenboak.com/2012/06/15/d3-flow-vis-tutorial.html&quot;&gt;flow visualization&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;SO question on &lt;a href=&quot;http://stackoverflow.com/questions/17656502/d3js-create-a-force-layout-with-fixed-nodes&quot;&gt;using a force layout&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;And some graph examples with D3:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Graph of &lt;a href=&quot;http://bl.ocks.org/mbostock/1153292&quot;&gt;Mobile Patent Suits&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;http://bl.ocks.org/lgersman/5311202&quot;&gt;State Diagram Editor&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;Simple &lt;a href=&quot;http://bl.ocks.org/rkirsling/5001347&quot;&gt;directed graph&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;Discussion of &lt;a href=&quot;http://www.is.kau.se/julioangulo/angulo/blog/?p=157568737&quot;&gt;force layout&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;http://toolongdidntread.com/graph-visualization/drawing-graphs-with-d3js-part-1/&quot;&gt;Drawing Graphs with D3&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</content>
 </entry>
 
 <entry>
   <title>Exploring Graphs with Neography</title>
   <link href="https://thinkingonthinking.com/Exploring-Graphs-With-Neography/"/>
   <updated>2014-02-05T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/Exploring-Graphs-With-Neography</id>
   <content type="html">&lt;p&gt;Working with graphs requires at least two things: Importing graph data and ‘traversing’ the graph. When connecting to Neo4J from Ruby, we also need some Ruby gem to talk to the graph engine. As Alberto Perdomo from &lt;a href=&quot;http://www.graphenedb.com/&quot;&gt;GrapheneDB&lt;/a&gt; pointed out, it is practical to start development in Ruby for graphs with &lt;a href=&quot;https://github.com/maxdemarzi/neography&quot;&gt;Neography&lt;/a&gt; by &lt;a href=&quot;https://twitter.com/maxdemarzi&quot;&gt;Max de Marzi&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;What do we need?&lt;/p&gt;

&lt;p&gt;First, I thought that I needed to run &lt;a href=&quot;http://jruby.org/&quot;&gt;JRuby&lt;/a&gt; - since Neo4J is Java, but since Neography is just a basic REST client, this is not stricly required.&lt;/p&gt;

&lt;p&gt;We can install Neography by adding this line into Gemfile:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;gem &apos;neography&apos;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Once, Neography is installed, we must setup our configs. When you work with a graph instance at &lt;a href=&quot;http://www.graphenedb.com/&quot;&gt;GrapheneDB&lt;/a&gt;, this will look like:&lt;/p&gt;

&lt;pre&gt;
Neography.configure do |config|
  config.protocol           = &quot;http://&quot;
  config.server             = &quot;yyy.xxx.zzz.graphenedb.com&quot;
  config.port               = 42424
  config.directory          = &quot;&quot; 
  config.cypher_path        = &quot;/cypher&quot;
  config.gremlin_path       = &quot;/ext/GremlinPlugin/graphdb/execute_script&quot;
  config.log_file           = &quot;neography.log&quot;
  config.log_enabled        = false
  config.slow_log_threshold = 0    # time in ms for query logging
  config.max_threads        = 20
  config.authentication     = &quot;basic&quot;  # &apos;basic&apos; or &apos;digest&apos;
  config.username           = &quot;some_user&quot;
  config.password           = &quot;some_password&quot;
  config.parser             = MultiJsonParser
end
&lt;/pre&gt;

&lt;p&gt;If we do this in e.g. ‘config/initializers/neography.rb’, we can simply use the Rails console to explore a graph. Let’s start this with:&lt;/p&gt;

&lt;pre&gt;
bundle exec rails c
&lt;/pre&gt;

&lt;p&gt;Next, we instantiate a Neography client with:&lt;/p&gt;

&lt;pre&gt;
irb(main):003:0&amp;gt; @neo = Neography::Rest.new
&lt;/pre&gt;

&lt;p&gt;This client can be used for creating nodes and relationships, as well as querying the graph. Basically, the discussion from Max de Marzi on &lt;a href=&quot;http://maxdemarzi.com/2012/01/04/getting-started-with-ruby-and-neo4j/&quot;&gt;Getting Started&lt;/a&gt; can be followed without problems.&lt;/p&gt;

&lt;p&gt;What is a bit of a problem is parsing the results from a Cypher query. For example, if we do:&lt;/p&gt;

&lt;pre&gt;
irb(main):002:0&amp;gt;  @neo.execute_query(&quot;START n=node(*) MATCH (n)-[r]-&amp;gt;(m) RETURN n,r,m;&quot;)
&lt;/pre&gt;

&lt;p&gt;We will see a huge hash containing meta information, many nodes and urls. It takes a bit of practice to convert this result set into something more “usable”. Here are some clues:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;The initial result set is a hash. Therefore we can call “.keys” on this and see a “columns” and “data” separation.&lt;/li&gt;
  &lt;li&gt;The interesting stuff is under the key “data”. So depending on the Cypher query, we have [“data”] we either an array or another hash.&lt;/li&gt;
  &lt;li&gt;Arrays can easily be explored with picking one element, e.g. with “first”.&lt;/li&gt;
  &lt;li&gt;Once, we understand what is in the result, we can use “map”, and rinse and repeat&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So, as an example, we can explore the name of all nodes with an relationship with the following query:&lt;/p&gt;

&lt;pre&gt;
@neo.execute_query(&quot;START n=node(*) MATCH (n)-[r]-&amp;gt;(m) RETURN n,r,m;&quot;)[&quot;data&quot;].map{|d| d.first[&quot;data&quot;]}
&lt;/pre&gt;

&lt;p&gt;And, we will see something like:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;=&amp;gt; [{&quot;name&quot;=&amp;gt;&quot;Max&quot;, &quot;age&quot;=&amp;gt;31}, {&quot;name&quot;=&amp;gt;&quot;Johnathan&quot;}, {&quot;name&quot;=&amp;gt;&quot;Johnathan&quot;}, {&quot;name&quot;=&amp;gt;&quot;Mark&quot;}, {&quot;name&quot;=&amp;gt;&quot;Mark&quot;}, {&quot;name&quot;=&amp;gt;&quot;Mark&quot;}, {&quot;name&quot;=&amp;gt;&quot;Mark&quot;}, {&quot;name&quot;=&amp;gt;&quot;Mark&quot;}, {&quot;name&quot;=&amp;gt;&quot;Mark&quot;}, {&quot;name&quot;=&amp;gt;&quot;Phil&quot;}, {&quot;name&quot;=&amp;gt;&quot;Phil&quot;}, {&quot;name&quot;=&amp;gt;&quot;Phil&quot;}, {&quot;name&quot;=&amp;gt;&quot;Phil&quot;}, {&quot;name&quot;=&amp;gt;&quot;Phil&quot;}, {&quot;name&quot;=&amp;gt;&quot;Phil&quot;}, {&quot;name&quot;=&amp;gt;&quot;Mary&quot;}, {&quot;name&quot;=&amp;gt;&quot;Mary&quot;}, {&quot;name&quot;=&amp;gt;&quot;Mary&quot;}, {&quot;name&quot;=&amp;gt;&quot;Mary&quot;}, {&quot;name&quot;=&amp;gt;&quot;Luke&quot;}, {&quot;name&quot;=&amp;gt;&quot;Luke&quot;}]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;So, with this, we have some way to run Graph queries from Ruby-on-Rails, although the processing of the result set needs some special wrappings. What are you learnings on querying graphs from Ruby? Especially, it would be great to hear more about the transformation from graph to JSON that can be rendered in a web application.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Small overview on API Authentication</title>
   <link href="https://thinkingonthinking.com/Small-Overview-on-API-authentication/"/>
   <updated>2014-02-02T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/Small-Overview-on-API-authentication</id>
   <content type="html">&lt;p&gt;In the context of &lt;a href=&quot;http://en.wikipedia.org/wiki/Representational_state_transfer#Key_goals&quot;&gt;REST&lt;/a&gt;, access control is mainly provided by setting credentials in HTTP headers. This is very nicely discussed in e.g. Amazon’s &lt;a href=&quot;http://s3.amazonaws.com/doc/s3-developer-guide/RESTAuthentication.html&quot;&gt;RESTAuthentication docs&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;There are many forms to pass tokens for authentication in distributed systems. Especially, the assumption that browsers are “dumb” (or unable to keep secure tokens), makes authentication difficult.&lt;/p&gt;

&lt;p&gt;A basic approach for authentication can be found in the HTTP specs (&lt;a href=&quot;http://www.ietf.org/rfc/rfc2616.txt&quot;&gt;RFC2616&lt;/a&gt; and &lt;a href=&quot;http://www.ietf.org/rfc/rfc2617.txt&quot;&gt;RFC2617&lt;/a&gt;).&lt;/p&gt;

&lt;p&gt;The specs are very dense, and can be read multiple times. So, to understand what is going on, let’s use try the following example with curl:&lt;/p&gt;

&lt;pre&gt;
curl http://example.com/some_action -H &apos;Authorization: Token token=&quot;79d4d9ee34e3f589ee94d080357afd8e&quot;
&lt;/pre&gt;

&lt;p&gt;What does this mean?&lt;/p&gt;

&lt;p&gt;In RFC2616, there is the following definition for a HTTP header field (section 14: “Header Field Definitions”):&lt;/p&gt;
&lt;pre&gt;
Authorization  = &quot;Authorization&quot; &quot;:&quot; credentials
&lt;/pre&gt;

&lt;p&gt;We need to look in RFC2617, to see what ‘credentials’ might be:&lt;/p&gt;

&lt;pre&gt;
credentials = auth-scheme #auth-param
&lt;/pre&gt;

&lt;p&gt;This can be interpreted as:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;‘auth-scheme’ specifies an authentication method. From RFC2617, this might be ‘Token’, ‘Basic’ or ‘Digest’, but it can be ‘AWS’ too, as in the case of Amazon.&lt;/li&gt;
  &lt;li&gt;‘auth-param’ is the actual authentication string. This part of the header field contains the sequence that grants access or should result into a 401 (Unauthorized) response.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In cases, such as connecting two web applications, token authentication is a popular scheme.  A client (= API consumer) passes a token to the server (e.g. a wrapper service for a database). The token from the request can then be used to grant access.&lt;/p&gt;

&lt;p&gt;For example, with a Rails based RESTful service, the API authentication could look like:&lt;/p&gt;

&lt;pre&gt;
def require_api_auth
  authenticate_or_request_with_http_token do |token, options|
    !(@current_user = User.where(access_token: token).first).nil?
  end
end
&lt;/pre&gt;

&lt;p&gt;Rails comes with these helpers built in (see &lt;a href=&quot;https://github.com/rails/rails/blob/27311fef5efa598f281649074255834546d2b4ec/actionpack/lib/action_controller/metal/http_authentication.rb#L389&quot;&gt;ActionController::Metal&lt;/a&gt;. But there might be cases where your API (or client) is a Rack application, and it’s actually some Rack methods too that are behind the helpers above, see &lt;a href=&quot;https://github.com/rails/rails/blob/27311fef5efa598f281649074255834546d2b4ec/actionpack/lib/action_controller/metal/http_authentication.rb#L429&quot;&gt;token_and_options&lt;/a&gt;. So, this check could easily be moved to Rack middleware.&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Connected Data</title>
   <link href="https://thinkingonthinking.com/Connected-Data/"/>
   <updated>2014-02-02T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/Connected-Data</id>
   <content type="html">&lt;p&gt;Editing a list of data is easy. Editing a list of data, where one item can be parent or child of others is quickly getting difficult.&lt;/p&gt;

&lt;p&gt;For example, take grouping skills into categories. The skill “Web Development” could be a child of “Software Development”, but also includes children such as HTML, Ruby-on-Rails or jQuery. Starting with a flat list of skills, a simple editor might support re-arranging skills and grouping “skills” into contexts.  After some fiddling, I came up with a simple user-interface to do this:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/static/images/skill_interface.png&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Since the connections between skills and categories is the main value behind the data, we just have created a graph problem. How would we best store our skill graph? The embedded version in JSON looks like:&lt;/p&gt;

&lt;p&gt;[{“id”:3,”children”:[{“id”:344},{“id”:25},{“id”:5},{“id”:200},{“id”:351},{“id”:26},{“id”:347},{“id”:37},{“id”:36},{“id”:19},{“id”:352},{“id”:354},{“id”:18},{“id”:4},{“id”:186},{“id”:35},{“id”:348}]},{“id”:346,”children”:[{“id”:34},{“id”:11},{“id”:353},{“id”:28},{“id”:10},{“id”:350},{“id”:33}]},{“id”:24,”children”:[{“id”:349},{“id”:2},{“id”:21},{“id”:13},{“id”:23},{“id”:14},{“id”:16},{“id”:22},{“id”:15},{“id”:17},{“id”:343}]},{“id”:20,”children”:[{“id”:31},{“id”:27}]},{“id”:29,”children”:[{“id”:12}]},{“id”:1,”children”:[{“id”:9},{“id”:8},{“id”:6},{“id”:32},{“id”:7},{“id”:30},{“id”:345}]}]&lt;/p&gt;

&lt;p&gt;This is neither easy to read, nor easy to store or to query, unless we would use a graph database. And, as first step, a graph database starts with a model:&lt;/p&gt;

&lt;p&gt;(User) -[:knows]-&amp;gt; (Skill) &amp;lt;-[:belongs_to]- (SkillCategory)&lt;/p&gt;

&lt;p&gt;Assuming this is our model, we still have a couple of questions: How to actually write the JSON from our editor to the skill graph? I think I want to explore &lt;a href=&quot;http://www.graphenedb.com/&quot;&gt;GrapheneDB&lt;/a&gt; - but I surely would need to integrate the service via some Rubygems, such as Neo4j.rb or Neography.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Share Your Infrastructure</title>
   <link href="https://thinkingonthinking.com/share_your_infrastructure/"/>
   <updated>2014-01-08T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/share_your_infrastructure</id>
   <content type="html">&lt;p&gt;A talk on Chef and Rubies&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Single-Page Applications with Thorax.js</title>
   <link href="https://thinkingonthinking.com/A-Blog-With-Thorax/"/>
   <updated>2014-01-02T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/A-Blog-With-Thorax</id>
   <content type="html">&lt;p&gt;Web applications with emphasis on interaction design are often labelled &lt;a href=&quot;http://en.wikipedia.org/wiki/Single-page_application&quot;&gt;Single-Page Application&lt;/a&gt; and change the way we think about &lt;a href=&quot;http://thinkingonthinking.com/Learning-From-Cinema-Websites/&quot;&gt;user experience&lt;/a&gt;, &lt;a href=&quot;http://thinkingonthinking.com/Here-Goes-Your-Web-Stack/&quot;&gt;application stacks&lt;/a&gt; and &lt;a href=&quot;http://thinkingonthinking.com/what-is-modular-architecture/&quot;&gt;web architecture&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Designing and developing applications that focus on interaction poses new challenges however. Some of these were summarized by &lt;a href=&quot;http://www.devthought.com/&quot;&gt;Guillermo Rauch&lt;/a&gt; in &lt;a href=&quot;https://cloudup.com/iuO7joTPdLr&quot;&gt;a great talk&lt;/a&gt; on optimistic UI and reactive interfaces at &lt;a href=&quot;http://redotheweb.com/2013/12/03/dotjs-2013-what-you-missed-what-I-missed.html&quot;&gt;dotJS&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;As we all know jQuery and maybe tried &lt;a href=&quot;http://backbonejs.org/&quot;&gt;Backbone.js&lt;/a&gt; already, let’s experiment with a framework for developing single-page applications on top of these libraries. Let’s look at &lt;a href=&quot;http://thoraxjs.org/&quot;&gt;Thorax.js&lt;/a&gt;. Why Thorax.js is awesome, is described by Peter de Croos &lt;a href=&quot;http://blog.peterdecroos.com/blog/2013/07/05/thorax-is-awesome/&quot;&gt;in this post&lt;/a&gt;. For the impatient folks, the demo of this blog post is: &lt;a href=&quot;http://thorax-blog.herokuapp.com/#&quot;&gt;Thorax Blog&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;getting-started&quot;&gt;Getting Started&lt;/h2&gt;

&lt;p&gt;Thorax comes out of the box with workflow automation based on Grunt, Yeoman and Bower. An overview on these approaches to frontend development are worked on in a &lt;a href=&quot;https://github.com/tooling/book-of-modern-frontend-tooling&quot;&gt;new book&lt;/a&gt; by Addy Osmani. However, to use Thorax, we don’t need an in-depth understanding of these tools, we just can use Yeoman and the &lt;a href=&quot;https://github.com/walmartlabs/generator-thorax/&quot;&gt;Thorax generator&lt;/a&gt;. The version of Thorax-generator as of this blog post is 0.0.7.&lt;/p&gt;

&lt;p&gt;To install the Thorax generator we do:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ npm install -g generator-thorax
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Next, we can scaffold a Thorax application with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ yo thorax blog
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;You will be asked, whether to use the “Chef’s Choice” application stack. This is a great name for &lt;a href=&quot;http://en.wikipedia.org/wiki/Convention_over_configuration&quot;&gt;“convention-over-configuration”&lt;/a&gt;, and we happily answer “yes”. You then can watch the dependencies getting installed, with the main results of obtaining a directory structure, some dependencies and a Gruntfile. With this, we are ready to go.&lt;/p&gt;

&lt;h2 id=&quot;mocking-data&quot;&gt;Mocking Data&lt;/h2&gt;

&lt;p&gt;Before hitting the application development, let’s prepare a data sandbox. In a real web application this sandbox would be replaced with an API, but since our goal is designing interfaces, we introduce an approach on mock data with &lt;a href=&quot;https://github.com/appendto/jquery-mockjax&quot;&gt;Mockjax&lt;/a&gt;. A similar approach to mock data on the client is currently also discussed in &lt;a href=&quot;http://discuss.emberjs.com/t/http-mocking-for-ember-testing/2846&quot;&gt;Ember.js with httprespond&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;To setup Mockjax, we include Mockjax in the Bower file with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&quot;jquery-mockjax&quot;: &quot;1.5.3&quot;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And run:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ bower install
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;We now include some Mockjax setup similar to &lt;a href=&quot;https://github.com/thopham/backbone-mockjax&quot;&gt;Backbone-Mockjax&lt;/a&gt;. In a file js/mocks.js, we include:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;define([&apos;jquery&apos;, &apos;mockjax&apos;], function($) {

  &apos;use strict&apos;;

  var mock = function() {

    $.ajaxSetup({
      dataType: &apos;json&apos;
    });

    $.mockjax({
       url: &apos;/api/posts&apos;,
       dataType: &apos;json&apos;,
       proxy: &apos;json/posts.json&apos;
    });
  };

  return {
    start: mock
  };
});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;To later use the mock setup in the application, we need to provide a reference in require-config.js (the development setups for run-time dependencies):&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&apos;mockjax&apos;: pathPrefix + &apos;bower_components/jquery-mockjax/jquery.mockjax&apos;

// ...

&apos;mockjax&apos;: {
   deps: [&apos;jquery&apos;]
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And, we create some mock data in json/posts.json:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; [ {&quot;id&quot;: 1, &quot;title&quot;: &quot;A Winter Day&quot;, &quot;body&quot;: &quot;On a cold winter day, ... &quot; }]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;With the mock in place, let’s tackle the application.&lt;/p&gt;

&lt;h2 id=&quot;setup-a-router&quot;&gt;Setup a router&lt;/h2&gt;

&lt;p&gt;The Thorax generator support scaffolding of all important application components. As such, let’s scaffold a Router. A Router in Thorax is very similar to a &lt;a href=&quot;http://backbonejs.org/#Router&quot;&gt;Backbone.Router&lt;/a&gt;:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ yo thorax:router posts
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;We should see:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;create js/routers/posts.js
create test/routers/posts.spec.js
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And, we can add the routes for browsing blog posts:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;define([
  &apos;backbone&apos;,
  &apos;mocks&apos;
], function (Backbone, Mock) {
  var collection;

  return Backbone.Router.extend({
    routes: {
      &quot;posts/:slug&quot;: &quot;showPost&quot;,
      &quot;&quot;: &quot;index&quot;
    },
    index: function() {
    },
    showPost: function() {
    }
  });
});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;We will basically setup two routes: One for the overview on blog posts and another for displaying single blog posts. We also introduce a reference to our Mock setup.
To have the router monitoring URL changes, we need to replace the HelloWorldRouter with the PostsRouter in main.js:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;require([
  &apos;jquery&apos;,
  &apos;backbone&apos;,
  &apos;views/root&apos;,
  &apos;routers/posts&apos;,
  &apos;helpers&apos;
], function ($, Backbone, RootView, PostsRouter) {
  $(function() {
    Backbone.history.start({
      pushState: false,
      root: &apos;/&apos;,
      silent: true
    });

    // RootView may use link or url helpers which
    // depend on Backbone history being setup
    // so need to wait to loadUrl() (which will)
    // actually execute the route
    RootView.getInstance(document.body);

    // Initialize your routers here
    new PostsRouter();

    // This will trigger your routers to start
    Backbone.history.loadUrl();
  });
});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This is all that needs to be done for the Router currently. We later will fill in the voids for the “index” and “showPost” routes.&lt;/p&gt;

&lt;h2 id=&quot;setup-a-posts-collection&quot;&gt;Setup a Posts collection&lt;/h2&gt;

&lt;p&gt;Once the Router is in place, it is time to setup a collection. Similarly to the Router, we can scaffold a Posts collection:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ yo thorax:collection posts
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Which results in:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;create js/collections/posts.js
create test/collections/posts.spec.js
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;We now setup the Collection, which is very similar to a &lt;a href=&quot;http://backbonejs.org/#Collection&quot;&gt;Backbone.Collection&lt;/a&gt;. We now provide the URL that is backed with Mockjax above:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;define([&apos;collection&apos;], function (Collection) {
  return Collection.extend({
    name: &apos;posts&apos;,
    url: &apos;/api/posts&apos;
  });
});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Let’s fetch the mock data from the browser as a first test. For this, we include the Collection as dependency in the Posts router. The router then becomes:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;define([
  &apos;backbone&apos;,
  &apos;mocks&apos;,
  &apos;collections/posts&apos;
], function (Backbone, Mock, PostsCollection) {
  var collection;

  if (!collection) {
    collection = new PostsCollection();
    Mock.start();
    console.log(&quot;*** Mock Posts API ***&quot;);
  }

  return Backbone.Router.extend({
    routes: {
      &quot;posts/:slug&quot;: &quot;showPost&quot;,
      &quot;&quot;: &quot;index&quot;
    },
    index: function() {
    },
    showPost: function() {
    }
  });
});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;First, we startup the application with Grunt:&lt;/p&gt;

&lt;p&gt;$ grunt&lt;/p&gt;

&lt;p&gt;Then, we can try that the collection works when we go to: http://0.0.0.0:8000/ and open the developer’s console:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; &amp;gt; T = require(&apos;thorax&apos;)
 &amp;gt; Posts = T.Collections.posts
 &amp;gt; posts = new Posts()
 &amp;gt; child {length: 0, models: Array[0], _byId: Object, cid: &quot;collection5&quot;, constructor: function…}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;When we now fetch data of the collection, we do:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; &amp;gt; posts.fetch()
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This should result in the browser into:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;MOCK GET: /api/posts 
Object {url: &quot;/api/posts&quot;, type: &quot;GET&quot;, isLocal: false, global: true, processData: true…}
 jquery.mockjax.js:546
Object {success: function, error: function, complete: function, reset: true}
XHR finished loading: &quot;http://0.0.0.0:8000/json/posts.json&quot;. 
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And the posts are populated with data:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;gt; posts.toJSON()
[
   Object
   body: &quot;On a cold winter day, ... &quot;
   id: 1
   title: &quot;A Winter Day&quot;
   __proto__: Object
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now, we can proceed to displaying blog posts with Views.&lt;/p&gt;

&lt;h2 id=&quot;rendering-posts&quot;&gt;Rendering Posts&lt;/h2&gt;

&lt;p&gt;Since Thorax.js supports &lt;a href=&quot;http://handlebarsjs.com/&quot;&gt;Handlebars.js&lt;/a&gt; and some view helpers, rendering is a piece of cake.&lt;/p&gt;

&lt;p&gt;First, we prepare the root view, which gives us a basic layout for the application. We use a basic layout derived from the HelloWorld example:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;lt;div class=&quot;header&quot;&amp;gt;
  &amp;lt;a href=&quot;#&quot;&amp;gt;Blog Demo&amp;lt;/a&amp;gt;
&amp;lt;/div&amp;gt;

&amp;lt;div class=&quot;thorax-container thorax-wrapper&quot;&amp;gt;
  &amp;lt;div class=&quot;thorax-primary&quot;&amp;gt;
    
  &amp;lt;/div&amp;gt;
&amp;lt;/div&amp;gt;

&amp;lt;div class=&quot;thorax-footer&quot;&amp;gt;
  &amp;lt;ul&amp;gt;
    &amp;lt;li&amp;gt;&amp;lt;a href=&quot;credits&quot;&amp;gt;Some credits&amp;lt;/a&amp;gt;&amp;lt;/li&amp;gt;
  &amp;lt;/ul&amp;gt;
&amp;lt;/div&amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Then, we scaffold views for the overview (index) of blog posts and showing a single blog post.&lt;/p&gt;

&lt;p&gt;This is done with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ yo thorax:view posts/index
$ yo thorax:view posts/show
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Then, in the js/views/posts/index.hbs we render the collection of Posts with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;lt;h2&amp;gt;The latest Posts&amp;lt;/h2&amp;gt;
{#collection}
  {#link &quot;posts/{id}&quot; expand-tokens=true}
    { title }
  {/link}
{/collection}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;In order to render the Posts, we have to connect the PostsView with the Posts collection in the Router with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;define([
  &apos;backbone&apos;,
  &apos;mocks&apos;,
  &apos;collections/posts&apos;,
  &apos;views/root&apos;,
  &apos;views/posts/index&apos;
], function (Backbone, Mock, PostsCollection, RootView, IndexView) {
  var collection;

  if (!collection) {
    collection = new PostsCollection();
    Mock.start();
    console.log(&quot;*** Mock Posts API ***&quot;);
  }

  return Backbone.Router.extend({
    routes: {
      &quot;posts/:slug&quot;: &quot;showPost&quot;,
      &quot;&quot;: &quot;index&quot;
    },
    index: function() {
      var view;
      collection.fetch({success: function(result) {
        view = new IndexView({collection: collection});
        console.log(collection.size());
        RootView.getInstance().setView(view);
      }});
    },
    showPost: function() {
    }
  });
});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;When we now go to the browser main page, we should see already a working index view. When we click on a blog post, nothing yet happens unless we add the rendering of showPost.&lt;/p&gt;

&lt;p&gt;So, we wire up the show view:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;showPost: function(id) {
  var post = collection.get(id);
  var view = new ShowPost({model: post});
  RootView.getInstance().setView(view);
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;To render a single post, we use this template:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&amp;lt;h2&amp;gt;{title}&amp;lt;/h2&amp;gt;
&amp;lt;hr&amp;gt;

&amp;lt;p&amp;gt;
  {body}
&amp;lt;/p&amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And we should have a running demo similar to: &lt;a href=&quot;http://thorax-blog.herokuapp.com/#&quot;&gt;Thorax Blog&lt;/a&gt;. You can find the source of this demo at &lt;a href=&quot;https://github.com/mulderp/thorax-blog&quot;&gt;Github&lt;/a&gt;.&lt;/p&gt;

&lt;h1 id=&quot;conclusions&quot;&gt;Conclusions&lt;/h1&gt;

&lt;p&gt;In this post, you touched the basics of a Single-Page Application with Thorax.js. You hopefully got an idea of mocking data, scaffolding application components (e.g. a Router, Collection and Views).&lt;/p&gt;

&lt;p&gt;There are still a number of interesting questions how to build applications with Thorax, and if you are interested to learn more, you might want to join the &lt;a href=&quot;https://groups.google.com/forum/?hl=en#!forum/thoraxjs&quot;&gt;Thorax Google group&lt;/a&gt; or leave feedback on this blog post (this is always highly appreciated). You also might want to follow &lt;a href=&quot;https://twitter.com/trombom&quot;&gt;Erik Trom&lt;/a&gt; or &lt;a href=&quot;https://twitter.com/colinmegill&quot;&gt;Colin Megill&lt;/a&gt; on Twitter who greatly helped me get going with Thorax. Also, thanks to Ryan Eastridge and &lt;a href=&quot;https://twitter.com/kpdecker&quot;&gt;Kevin Decker&lt;/a&gt; for their efforts on Thorax in the first place!&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Business Model Development</title>
   <link href="https://thinkingonthinking.com/business-model-development/"/>
   <updated>2013-12-22T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/business-model-development</id>
   <content type="html">&lt;p&gt;How to achieve &lt;a href=&quot;http://thinkingonthinking.com/Elegance-in-a-software-business/&quot;&gt;elegance in software products&lt;/a&gt;, and how to avoid the programming trap that &lt;a href=&quot;https://twitter.com/thomasfuchs&quot;&gt;@thomasfuchs&lt;/a&gt; mentions:&lt;/p&gt;

&lt;blockquote class=&quot;twitter-tweet&quot; lang=&quot;en&quot;&gt;&lt;p&gt;Priorities of many programmers:&amp;#10;1. New frameworks, 2. Clean code, 3. Reinvent wheels&amp;#10;&amp;#10;How it should be:&amp;#10;1, 2 &amp;amp; 3: Make users awesome&lt;/p&gt;&amp;mdash; Thomas Fuchs ⛄ (@thomasfuchs) &lt;a href=&quot;https://twitter.com/thomasfuchs/statuses/411123286027026432&quot;&gt;December 12, 2013&lt;/a&gt;&lt;/blockquote&gt;
&lt;script async=&quot;&quot; src=&quot;//platform.twitter.com/widgets.js&quot; charset=&quot;utf-8&quot;&gt;&lt;/script&gt;

&lt;p&gt;Clearly, if we work on bringing empathy into software products, we need some other tools to manage creativity than those found in programming.&lt;/p&gt;

&lt;p&gt;Here are some tools, that I find helpful:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;Business Model Canvas: Users and their needs are the fundamental driver behind any (software) product. The Business Model Canvas by &lt;a href=&quot;http://businessmodelalchemist.com/&quot;&gt;Alex Osterwalder&lt;/a&gt; is a great approach to explore ideas around needs, product ideas and the context. In exploring a business model canvas, you also might want to look at &lt;a href=&quot;http://businessmodelhub.com/&quot;&gt;http://businemodelhub.com&lt;/a&gt; too.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Lean Philosophy: In the Japanese school of innovation, an important principle is &lt;a href=&quot;http://en.wikipedia.org/wiki/Genchi_Genbutsu&quot;&gt;“go and see”&lt;/a&gt;. Start with the users and their contexts. A related approach is the “Customer Value Map”, or “Empathy Map”, described by Alex Osterwalder, &lt;a href=&quot;http://businessmodelalchemist.com/blog/2012/01/the-customer-value-canvas-v-0-8.html&quot;&gt;here&lt;/a&gt; or the &lt;a href=&quot;http://www.gogamestorm.com/?p=42&quot;&gt;“Empathy Map”&lt;/a&gt;. For this, we need to talk with our customers and experience the same as they do. This is topic of many discussions, also on the recent &lt;a href=&quot;http://www.skmurphy.com/blog/2013/12/14/lean-startup-conference-2013-roundup/&quot;&gt;“Lean Startup” conference&lt;/a&gt;&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Once you work with a canvas, and have ideas on a map, I found a quote by &lt;a href=&quot;https://github.com/coffeemug/&quot;&gt;Coffeemug&lt;/a&gt;, interesting: &lt;a href=&quot;https://news.ycombinator.com/item?id=2080727&quot;&gt;“Pick a market you care about, and the best product in that market. Then think for a week how you could improve that product by an order of magnitude. List ideas. Throw them out. List more. When we’re talking about disruptive improvement, thinking about it in terms of “orders of magnitude” is hard, so try to think about it in terms of what huge, meaningful thing the users could do that they can’t do now.”&lt;/a&gt;&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;When it comes to planning a software product, I like the &lt;a href=&quot;http://www.romanpichler.com/blog/agile-product-innovation/the-product-canvas/&quot;&gt;Product Canvas&lt;/a&gt; by Roman Pichler. Here, we are putting less focus on the constraints of a business, but more on the actual short-terms goals, and the expected outcomes for a user. This might help us in planning specific outcomes for users.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Playing with the above ideas is helped a lot by &lt;a href=&quot;http://bmfiddle.com/&quot;&gt;Bmfiddle&lt;/a&gt; - an online tool for exploring the Business Model Canvasses, Empathy Maps, and Product Canvasses. If you need help to plan your product strategy, shoot me an &lt;a href=&quot;mailto:mulder.patrick@gmail.com&quot;&gt;email&lt;/a&gt; or leave feedback below.&lt;/p&gt;

&lt;p&gt;Happy brainstorming!&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Elegance in a software business</title>
   <link href="https://thinkingonthinking.com/Elegance-in-a-software-business/"/>
   <updated>2013-12-15T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/Elegance-in-a-software-business</id>
   <content type="html">&lt;p&gt;Let’s structure some thoughts about running a business - a topic which I touched in a &lt;a href=&quot;http://thinkingonthinking.com/fashion-and-business-models/&quot;&gt;previous post&lt;/a&gt;. In a networking world, designing interactions is ever more important. Our businesses require &lt;a href=&quot;http://thinkingonthinking.com/Push-Conference/&quot;&gt;a kind of creativity&lt;/a&gt; that make us feel human and special again.&lt;/p&gt;

&lt;p&gt;For this, as &lt;a href=&quot;http://www.ted.com/talks/john_maeda_how_art_technology_and_design_inform_creative_leaders.html&quot;&gt;John Maeda&lt;/a&gt; - one of my role models - explains, we join artists and designers with their quest for empathy, as well as engineers with their drive for efficiencies. Our goals too must become “elegance”.&lt;/p&gt;

&lt;p&gt;But what is elegance? And, what is elegance in modern IT companies? Let’s look at some examples:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;The blog &lt;a href=&quot;http://37signals.com/svn&quot;&gt;Signal-vs-Noise&lt;/a&gt; of 37signals is elegant. Here, technology is put in the background, to show how we work and live as humans in the foreground. The logo of 37signals is just great, as are many finer design details and the content.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;http://www.dataverse.me/&quot;&gt;Dataverse&lt;/a&gt; looks elegant to me, since it asks us to “re-imagine our data.” - so, we realize we have a fundamental problem with data today. This small sentence somehow embraces empathy to me. It picks up one of my problems instead of showing their offerings directly.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;For example, &lt;a href=&quot;http://www.neo.com/&quot;&gt;neo.com&lt;/a&gt; puts out the claim to combine “engineering, lean thinking and delightful design.” The company thereby touches our emotions that make the difference when making, validating and educating businesses. Also, it shows how design can slip into software companies with the concept of “lean”, job stories, storyboards and mood boards.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;http://cospired.com/&quot;&gt;cospired&lt;/a&gt; does a good job addressing our imagination too. The word creation derived from “inspiration” is just great, and I like the pentagons in the background, reminding me on the complexity of &lt;a href=&quot;http://en.wikipedia.org/wiki/Carbon_ring&quot;&gt;carbon rings&lt;/a&gt; in chemistry and addresses the scientific nature of software development.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Not a software company directly, &lt;a href=&quot;http://artsy.net/&quot;&gt;art.sy&lt;/a&gt; makes a living of communicating questions of art to a wider audience. It is another form where engineering and design meet.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So, what are your personal examples of companies that embrace empathy, elegance or inspiration?&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Chef and Rubies</title>
   <link href="https://thinkingonthinking.com/Chef-and-Rubies/"/>
   <updated>2013-12-11T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/Chef-and-Rubies</id>
   <content type="html">&lt;p&gt;&lt;a href=&quot;http://docs.opscode.com/&quot;&gt;Chef&lt;/a&gt; is a tool for server automation - built with Ruby. When you provision a virtual machine, and you need to have your own Ruby version, Chef’s Ruby and your application’s Ruby, can bite each other. So, let’s look again how to use the cookbook wrappers for &lt;a href=&quot;https://github.com/fnichol/chef-rvm&quot;&gt;rvm&lt;/a&gt; and &lt;a href=&quot;https://github.com/fnichol/chef-rbenv&quot;&gt;rbenv&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;fetching-dependencies&quot;&gt;Fetching Dependencies&lt;/h2&gt;

&lt;p&gt;Chef is built around cookbooks, and you can fetch cookbook dependencies for a project with Berkshelf, as explained in &lt;a href=&quot;http://thinkingonthinking.com/berkshelf-and-chef/&quot;&gt;a previous post&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;To fetch the RVM cookbook, we use this Berksfile:&lt;/p&gt;

&lt;pre&gt;
  site :opscode
  
  cookbook &apos;rvm&apos;, :git =&amp;gt; &quot;https://github.com/fnichol/chef-rvm&quot;
&lt;/pre&gt;

&lt;p&gt;Now, we do:&lt;/p&gt;

&lt;pre&gt;
  berks install
&lt;/pre&gt;

&lt;h2 id=&quot;pre-provision-the-vm&quot;&gt;Pre-provision the VM&lt;/h2&gt;

&lt;p&gt;As mentioned, Chef is built with Ruby. but in many cases there will be no Ruby at all on a fresh VM. Therefore, we need to find some other way to bring Chef to the VM, without using a cookbook.&lt;/p&gt;

&lt;p&gt;The fastest way to get a Chef installation on a new VM is by using a shell provisioning step:&lt;/p&gt;

&lt;pre&gt;
  config.vm.provision :shell, :inline =&amp;gt; &quot;/usr/bin/apt-get update --fix-missing&quot;
  config.vm.provision :shell, :inline =&amp;gt; &quot;sudo apt-get install -y curl; curl -L https://www.opscode.com/chef/install.sh | sudo bash&quot;
&lt;/pre&gt;

&lt;p&gt;When we now provision the VM, with:&lt;/p&gt;

&lt;pre&gt;
  $ vagrant up
&lt;/pre&gt;

&lt;p&gt;or:&lt;/p&gt;

&lt;pre&gt;
  $ vagrant provision
&lt;/pre&gt;

&lt;p&gt;We will run a Chef installation, and obtain a Ruby in /opt/chef/embedded&lt;/p&gt;

&lt;h2 id=&quot;installing-the-ruby-wrapper&quot;&gt;Installing the Ruby wrapper&lt;/h2&gt;

&lt;p&gt;As next step, we must configure the Ruby wrapper. Normally, we define parameters for cookbooks in a &lt;a href=&quot;https://github.com/mulderp/chef-rails-stack/blob/rails4_stack/node.json&quot;&gt;JSON file&lt;/a&gt;, but as we want to have the “fastest” way to get some Ruby on a VM, we do it “inline” in Vagrant.&lt;/p&gt;

&lt;p&gt;The configuration in Vagrant could look like:&lt;/p&gt;

&lt;pre&gt;
  config.vm.provision :chef_solo do |chef|
    chef.cookbooks_path = &quot;./cookbooks&quot;
    chef.add_recipe &quot;rvm::system&quot;
    chef.add_recipe &quot;rvm::vagrant&quot;

    chef.json = {
      &apos;rvm&apos; =&amp;gt; {
         &apos;rubies&apos; =&amp;gt; [&quot;2.0.0-p353&quot;],
         &apos;default_ruby&apos; =&amp;gt; &quot;2.0.0-p353&quot;,
         :vagrant =&amp;gt; { :system_chef_solo =&amp;gt; &quot;/opt/chef/bin/chef-solo&quot; }
      }
    }

    chef.log_level = :debug
  end
&lt;/pre&gt;

&lt;p&gt;There are 3 “tricks” here:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;We must run the “rvm::system” recipe to get the rvm cookbook actually installing a Ruby&lt;/li&gt;
  &lt;li&gt;We include the recipe “rvm::vagrant” to get the setup of groups right&lt;/li&gt;
  &lt;li&gt;We setup a wrapper for the original Chef Ruby, since we would loose the “chef-solo” Ruby gem otherwise&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Well, that should be it. Happy cooking.&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Bookshelf - An ORM for JavaScript</title>
   <link href="https://thinkingonthinking.com/Bookshelf-an-ORM-for-Node/"/>
   <updated>2013-12-08T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/Bookshelf-an-ORM-for-Node</id>
   <content type="html">&lt;p&gt;“We want to be able to ask questions to our data, without predefining the way we structure our data” - an argument for “No”-SQL, put forward in this very nice &lt;a href=&quot;http://www.youtube.com/watch?v=fEsmydn747c&quot;&gt;talk&lt;/a&gt; on ElasticSearch by Shay Banon.&lt;/p&gt;

&lt;p&gt;Still, today we often use SQL, and it is required that you predefine your schema or structure your data beforehand. This means to query data, you must add and remove attributes from a schema, define how data is connected (or how data can be joined) to run actual queries.&lt;/p&gt;

&lt;p&gt;And if you want to do this with Node.js, you often were left alone with your RDBMS backend. Not anymore, since thanks to &lt;a href=&quot;https://twitter.com/tgriesser&quot;&gt;Tim Griesser&lt;/a&gt;, there are a number of fresh ideas for working with data in Node.js. Let’s have a look at &lt;a href=&quot;http://knexjs.org/&quot;&gt;Knex&lt;/a&gt; and &lt;a href=&quot;http://bookshelfjs.org/&quot;&gt;Bookshelf&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;knex&quot;&gt;Knex&lt;/h2&gt;

&lt;p&gt;&lt;a href=&quot;http://knexjs.org/&quot;&gt;Knex&lt;/a&gt; is a promissing wrapper for SQL in JavaScript. With Knex, we quickly can connect to a MySQL, Postgres or SQLite database.&lt;/p&gt;

&lt;p&gt;As example, let’s start with &lt;a href=&quot;https://sqlite.org/download.html&quot;&gt;SQLite3&lt;/a&gt; database that comes as default in a knexfile.js:&lt;/p&gt;

&lt;pre&gt;
/ knexfile.js

module.exports = {

  development: {
    client: &apos;sqlite3&apos;,
    connection: {
      filename: &apos;./mydb.sqlite3&apos;
    }
  },
&lt;/pre&gt;

&lt;p&gt;In knex migrations, you can use the knex object to define tables and attributes. Basically, you can do &lt;a href=&quot;http://knexjs.org/#Schema&quot;&gt;schema operations&lt;/a&gt;:&lt;/p&gt;

&lt;pre&gt;
// migrations/20161013223537_createMovies.js
var table = function (table) {
  table.string(&apos;title&apos;);
  table.timestamps();
};

exports.up = function(knex, Promise) {

  return knex.schema.createTable(&apos;movies&apos;, table).then(function () {
    console.log(&apos;Movies Table is Created!&apos;);
  });
};

exports.down = function(knex, Promise) {

  return knex.schema.dropTable(&apos;movies&apos;).then(function() {
    console.log(&apos;Movies Table is dropped!&apos;);
  });
};
&lt;/pre&gt;

&lt;p&gt;Adding a new column to the movies table is as easy with the &lt;a href=&quot;http://knexjs.org/#Schema-table&quot;&gt;table&lt;/a&gt; commands:&lt;/p&gt;

&lt;pre&gt;
// migrations/20161013224455_addYear.js
var addYear = function (table) {
    table.integer(&apos;year&apos;);
};


function rmYear (table) {
    table.dropColumn(&apos;year&apos;);
}


exports.up = function(knex, Promise) {
  return knex.schema.table(&apos;movies&apos;, addYear)
    .then(function() { console.log(&quot;Attribute added&quot;); })
      .catch(function(err) { console.log(err) });

};

exports.down = function(knex, Promise) {
  return knex.schema.table(&apos;movies&apos;, rmYear)
    .then(function() { console.log(&quot;Attribute removed&quot;); })
      .catch(function(err) { console.log(err) });
};
&lt;/pre&gt;

&lt;p&gt;Last, you need &lt;a href=&quot;http://stackoverflow.com/questions/28869148/knex-js-db-call-does-not-complete&quot;&gt;to close the connection&lt;/a&gt;. This is done by adding on the last statement:&lt;/p&gt;

&lt;pre&gt;
  .finally(function() {
    DB.destroy();
  });
&lt;/pre&gt;

&lt;p&gt;Now, after you run the script, you can check the simple schema with SQLite:&lt;/p&gt;

&lt;pre&gt;
sqlite&amp;gt; .schema movies
CREATE TABLE &quot;movies&quot; (&quot;title&quot; varchar(255), &quot;created_at&quot; datetime, &quot;updated_at&quot; datetime, &quot;year&quot; integer);
&lt;/pre&gt;

&lt;h2 id=&quot;insert-data&quot;&gt;Insert data&lt;/h2&gt;

&lt;p&gt;Next, let’s insert some data:&lt;/p&gt;

&lt;pre&gt;
DB(&apos;movies&apos;)
   .insert({title: &quot;The Artist&quot;, year: 2010})
   .then(function() { console.log(&quot;added&quot;); })
   .catch(function(err) { console.log(err) });
&lt;/pre&gt;

&lt;p&gt;And, when we check in SQLite3:&lt;/p&gt;

&lt;pre&gt;
sqlite&amp;gt; select * from movies;
The Artist|||2010
&lt;/pre&gt;

&lt;p&gt;Ok, so we have Create and Read, I leave the Update and Delete as an exercise to the reader. It would be great to hear your CRUD experience.&lt;/p&gt;

&lt;p&gt;The example so far is at &lt;a href=&quot;https://github.com/mulderp/bookshelf-demo&quot;&gt;this github repo&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;bookshelf&quot;&gt;Bookshelf&lt;/h2&gt;

&lt;p&gt;Although SQL is nice for asking questions to data (and once in a while to write new data), the applications are based on data abstractions. This is where the realm of ORM (or object-relational-mappers) starts. Or, &lt;a href=&quot;http://bookshelfjs.org/&quot;&gt;Bookshelf&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Inspired by Backbone and Underscore.js, Bookshelf provides a Model and Collection abstraction. Similarly as in Backbone, a model is wrapping data attributes, or a row in an RDBMS. The important idea in Bookshelf however, is a standarized way to write, read and delete data: “save”, “fetch” and “delete”. Additionally, Bookshelf let’s you make connections between data or “joins”.&lt;/p&gt;

&lt;p&gt;We could write, e.g. a Movie hasMany Genres, as well as Genres belongsToMany Movies.&lt;/p&gt;

&lt;p&gt;Another idea, that I like is the “&lt;a href=&quot;http://bookshelfjs.org/#Collection-forge&quot;&gt;forge&lt;/a&gt;” method. With this, you can set default values in a model. That can make a lot of sense, if you store predefined values, such as Mrs. or Mr., or a default date for example.&lt;/p&gt;

&lt;p&gt;The Collections in Bookshelf look great too, since they give you a lot of helpers from Underscore. I need to explore these a bit with e.g. 1000 rows of movie data, and see how good they work in practice, but the idea looks great. Another blog post follows hopefully soon.&lt;/p&gt;

&lt;h2 id=&quot;conclusions&quot;&gt;Conclusions&lt;/h2&gt;

&lt;p&gt;Knex and Bookshelf provide a very nice set of server-side abstractions to work with a RDBMS in Node.js. This makes your data-layer more flexible, and might also be interesting in a schema-less world, as this &lt;a href=&quot;https://github.com/tgriesser/bookshelf/issues/102&quot;&gt;discussion&lt;/a&gt; and the mentioning of mongoose in &lt;a href=&quot;https://github.com/tgriesser/bookshelf/issues/69&quot;&gt;this discussion&lt;/a&gt; shows.&lt;/p&gt;

&lt;p&gt;Also, I wonder, if Knex and Bookshelf could be used to develop an API application with &lt;a href=&quot;https://www.arangodb.org/foxx&quot;&gt;Foxx&lt;/a&gt; and &lt;a href=&quot;https://www.arangodb.org/&quot;&gt;ArangoDB&lt;/a&gt;. ArangoDB comes with &lt;a href=&quot;https://www.arangodb.org/manuals/current/Aql.html&quot;&gt;AQL&lt;/a&gt; - a query language for NoSQL that supports join operations. One promise of a combination of both ArangoDB and Bookshelf might be to ask questions to data, without predefining a schema.&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Berkshelf and Chef</title>
   <link href="https://thinkingonthinking.com/berkshelf-and-chef/"/>
   <updated>2013-12-07T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/berkshelf-and-chef</id>
   <content type="html">&lt;p&gt;The last blog post on &lt;a href=&quot;http://thinkingonthinking.com/A-Data-Platform-in-15-minutes/&quot;&gt;A Data Platform in 15 minutes&lt;/a&gt; got quite some attention.&lt;/p&gt;

&lt;p&gt;While writing that post, I found that a blog post on a “single” topic is very useful in the long term. And it is better to avoid an “overview” when writing, such as &lt;a href=&quot;http://thinkingonthinking.com/A-custom-rails-stack-with-chef/&quot;&gt;this old post&lt;/a&gt;, where I mix Chef, Berkshelf, RVM and Ruby.&lt;/p&gt;

&lt;p&gt;So, to practice writing, I want to review some basics of Chef. Imagine we have a fresh VM (maybe from the &lt;a href=&quot;https://github.com/opscode/bento&quot;&gt;Bento&lt;/a&gt; image repository from Opscode). And, we want to quickly install a recent version of Node.js.&lt;/p&gt;

&lt;h2 id=&quot;berkshelf&quot;&gt;Berkshelf&lt;/h2&gt;

&lt;p&gt;A fast way to achieve that would be using &lt;a href=&quot;https://github.com/berkshelf/vagrant-berkshelf&quot;&gt;vagrant-berkshelf&lt;/a&gt; . In contrast to my older posts on Chef, Berkshelf now must be installed as plugin like this:&lt;/p&gt;

&lt;pre&gt;
$ vagrant plugin install vagrant-berkshelf
Installing the &apos;vagrant-berkshelf&apos; plugin. This can take a few minutes...
Installed the plugin &apos;vagrant-berkshelf (1.3.4)&apos;!
&lt;/pre&gt;

&lt;p&gt;Now, we can easily fetch cookbook dependencies, e.g. for Node.js, with this Berksfile:&lt;/p&gt;

&lt;pre&gt;
site :opscode

cookbook &apos;apt&apos;
cookbook &apos;node&apos;
&lt;/pre&gt;

&lt;p&gt;However, if we try to &lt;a href=&quot;http://docs-v1.vagrantup.com/v1/docs/provisioners.html&quot;&gt;provision&lt;/a&gt;&lt;/p&gt;

&lt;pre&gt;
  config.berkshelf.enabled = true
  config.vm.provision :chef_solo do |chef|
    chef.cookbooks_path = &quot;./cookbooks&quot;
    chef.add_recipe &quot;apt&quot;
    chef.add_recipe &quot;node&quot;
  end
&lt;/pre&gt;

&lt;p&gt;and&lt;/p&gt;

&lt;pre&gt;
$ vagrant provision
&lt;/pre&gt;

&lt;p&gt;We will hit an error, that chef-solo is missing. Let’s look in the next section at some option.&lt;/p&gt;

&lt;h2 id=&quot;provision-the-vm&quot;&gt;Provision the VM&lt;/h2&gt;

&lt;p&gt;Debugging Chef and cookbooks often requires an ssh login, and looking at the actual logfiles of the problem. We can do this:&lt;/p&gt;

&lt;pre&gt;
$ vagrant ssh
&lt;/pre&gt;

&lt;p&gt;And try:&lt;/p&gt;

&lt;pre&gt;
$ chef-solo
&lt;/pre&gt;

&lt;p&gt;Indeed, the provisioner is missing. What can we do? First, let’s try:&lt;/p&gt;

&lt;pre&gt;
$ gem install chef
&lt;/pre&gt;

&lt;p&gt;This will complain about missing permissions. So, we have to do a:&lt;/p&gt;

&lt;pre&gt;
$ sudo gem install chef
&lt;/pre&gt;

&lt;p&gt;Then, we get something more running, but we will hit some missing system dependencies for the chef gem. So, let’s try to:&lt;/p&gt;

&lt;pre&gt;
sudo aptitude install ruby1.9.1-dev
&lt;/pre&gt;

&lt;p&gt;And, then again the “sudo gem install chef”. After seeing the installation succeed, the provisioner should be ready to be called from Vagrant.&lt;/p&gt;

&lt;p&gt;So, we can do a:&lt;/p&gt;

&lt;pre&gt;
$ vagrant up
Bringing machine &apos;default&apos; up with &apos;virtualbox&apos; provider...
[default] Clearing any previously set forwarded ports...
[Berkshelf] This version of the Berkshelf plugin has not been fully tested on this version of Vagrant.
[Berkshelf] You should check for a newer version of vagrant-berkshelf.
[Berkshelf] If you encounter any errors with this version, please report them at https://github.com/RiotGames/vagrant-berkshelf/issues
[Berkshelf] You can also join the discussion in #berkshelf on Freenode.
[Berkshelf] Updating Vagrant&apos;s berkshelf: &apos;/Users/pmu/.berkshelf/default/vagrant/berkshelf-20131207-29173-1fwx1d8-default&apos;
[Berkshelf] Using apt (1.9.0)
[Berkshelf] Using node (1.0.1)
[Berkshelf] Using git (2.7.0)
[Berkshelf] Using dmg (2.0.8)
[Berkshelf] Using build-essential (1.4.2)
[Berkshelf] Using yum (2.4.4)
[Berkshelf] Using windows (1.11.0)
[Berkshelf] Using chef_handler (1.1.4)
[Berkshelf] Using runit (1.4.0)
[default] Creating shared folders metadata...
[default] Clearing any previously set network interfaces...
[default] Preparing network interfaces based on configuration...
[default] Forwarding ports...
[default] -- 22 =&amp;gt; 2222 (adapter 1)
[default] -- 8529 =&amp;gt; 5000 (adapter 1)
[default] Booting VM...
[default] Waiting for machine to boot. This may take a few minutes...
[default] Machine booted and ready!
[default] Mounting shared folders...
[default] -- /vagrant
[default] -- /tmp/vagrant-chef-1/chef-solo-1/cookbooks

$ vagrant provision
[Berkshelf] This version of the Berkshelf plugin has not been fully tested on this version of Vagrant.
[Berkshelf] You should check for a newer version of vagrant-berkshelf.
[Berkshelf] If you encounter any errors with this version, please report them at https://github.com/RiotGames/vagrant-berkshelf/issues
[Berkshelf] You can also join the discussion in #berkshelf on Freenode.
[Berkshelf] Updating Vagrant&apos;s berkshelf: &apos;/Users/pmu/.berkshelf/default/vagrant/berkshelf-20131207-29173-1fwx1d8-default&apos;
[Berkshelf] Using apt (1.9.0)
[Berkshelf] Using node (1.0.1)
[Berkshelf] Using git (2.7.0)
[Berkshelf] Using dmg (2.0.8)
[Berkshelf] Using build-essential (1.4.2)
[Berkshelf] Using yum (2.4.4)
[Berkshelf] Using windows (1.11.0)
[Berkshelf] Using chef_handler (1.1.4)
[Berkshelf] Using runit (1.4.0)
[default] Running provisioner: chef_solo...
Generating chef JSON and uploading...
Running chef-solo...
stdin: is not a tty
[2013-12-07T08:06:47+00:00] INFO: Forking chef instance to converge...
[2013-12-07T08:06:47+00:00] INFO: *** Chef 11.8.2 ***
[2013-12-07T08:06:47+00:00] INFO: Chef-client pid: 1097
[2013-12-07T08:06:48+00:00] INFO: Setting the run_list to [&quot;recipe[apt]&quot;, &quot;recipe[node]&quot;] from JSON
[2013-12-07T08:06:48+00:00] INFO: Run List is [recipe[apt], recipe[node]]
[2013-12-07T08:06:48+00:00] INFO: Run List expands to [apt, node]
[2013-12-07T08:06:48+00:00] INFO: Starting Chef Run for vagrant.vm
[2013-12-07T08:06:48+00:00] INFO: Running start handlers
[2013-12-07T08:06:48+00:00] INFO: Start handlers complete.
[2013-12-07T08:06:59+00:00] INFO: execute[apt-get-update] ran successfully
[2013-12-07T08:07:02+00:00] INFO: package[update-notifier-common] sending run action to execute[apt-get-update] (immediate)
[2013-12-07T08:07:07+00:00] INFO: execute[apt-get-update] ran successfully
[2013-12-07T08:07:07+00:00] INFO: directory[/var/cache/local] created directory /var/cache/local
[2013-12-07T08:07:07+00:00] INFO: directory[/var/cache/local] owner changed to 0
....
&lt;/pre&gt;

&lt;p&gt;And the run looks successfull.&lt;/p&gt;

&lt;p&gt;After a while, Node should be ready:&lt;/p&gt;

&lt;pre&gt;
vagrant@vagrant:~$ node -v
v0.11.10-pre
vagrant@vagrant:~$ npm -v
1.3.15
&lt;/pre&gt;

&lt;h2 id=&quot;iterate-and-improve&quot;&gt;Iterate and improve&lt;/h2&gt;

&lt;p&gt;Now, we could spin-off a Node powered VM unlimited times, with only some commands. However, there is one place, in the protocol above that might be improved. The manual login and installation of the chef provisioner.  A quick googling gave a link that looks promising by Joshua Timberman, a well known contributor in the field of server automation. So, in &lt;a href=&quot;http://jtimberman.housepub.org/blog/2012/08/10/vagrant-multiple-provisioners/&quot;&gt;this post&lt;/a&gt;, he discusses how to use multiple provisioners.&lt;/p&gt;

&lt;p&gt;An idea would now be to replace the manual steps above by a shell script, and execute those for the actual Chef provisioning run. I leave that as an exercise up to the reader. If anyone wants to report back how it goes, that would be great! Thanks.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>A Data Platform in 15 Minutes</title>
   <link href="https://thinkingonthinking.com/A-Data-Platform-in-15-minutes/"/>
   <updated>2013-12-03T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/A-Data-Platform-in-15-minutes</id>
   <content type="html">&lt;p&gt;Classically, the essence of the web has been connecting files and documents in a network. In many cases, these files and documents are representations of resources (or data). And with the rise of Apple and Google, the way we interact with those remote resources is challenging us as software engineers every day a bit more.&lt;/p&gt;

&lt;p&gt;So, in this post, I want to show a way, how to setup a node that acts as data repository, and which can be used as backend for the development and for the production of so called single-page applications, or JavaScript applications.&lt;/p&gt;

&lt;h2 id=&quot;1-a-vagrant-basebox&quot;&gt;1. A Vagrant basebox&lt;/h2&gt;

&lt;p&gt;As first step, let’s set a node. Vagrant is a great wrapper for fetching, starting and stopping virtual machines (VMs), and I have blogged on advanced VM use cases with Chef automation &lt;a href=&quot;http://thinkingonthinking.com/building-a-Vagrant-base-box/&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;http://thinkingonthinking.com/building-infrastructure/&quot;&gt;here&lt;/a&gt;. With Vagrant, we can do a number of learning before going to a hosted service such as Amazon EC2 or another provider of cloud resources.&lt;/p&gt;

&lt;p&gt;Opscode’s Bento project is a great place to find baseboxes, so, let’s fetch a standard Ubuntu “precise” box:&lt;/p&gt;

&lt;pre&gt;
  vagrant box add ubuntu-12 http://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_ubuntu-12.04_chef-provisionerless.box
&lt;/pre&gt;

&lt;p&gt;Downloading the Ubuntu image will take a while. So, depending on your network speed, you should come back in 10 minutes now.&lt;/p&gt;

&lt;p&gt;Once the base box is downloaded, we create a new project on our development machine (or “host”):&lt;/p&gt;

&lt;pre&gt;
  $ mkdir movies_repo
  $ cd movies_repo
  $ vagrant init ubuntu-12
&lt;/pre&gt;

&lt;p&gt;This gives us a Vagrantfile, i.e. a configuration for the virtual machine.&lt;/p&gt;

&lt;p&gt;We can tune this in all kinds of directions, but right now, we only want port forwarding from the VM to our host. So, we add in our Vagrantfile:&lt;/p&gt;

&lt;pre&gt;
  config.vm.network :forwarded_port, guest: 8529, host: 5000
&lt;/pre&gt;

&lt;p&gt;The guest port 8529 is the ArangoDB port, which we turn to in a second.&lt;/p&gt;

&lt;p&gt;Now, we start the virtual machine with the famous &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;vagrant up&lt;/code&gt; command:&lt;/p&gt;

&lt;pre&gt;
   $ vagrant up
&lt;/pre&gt;

&lt;p&gt;And we will see something like:&lt;/p&gt;

&lt;pre&gt;
Bringing machine &apos;default&apos; up with &apos;virtualbox&apos; provider...
[default] Clearing any previously set forwarded ports...
[default] Creating shared folders metadata...
[default] Clearing any previously set network interfaces...
[default] Preparing network interfaces based on configuration...
[default] Forwarding ports...
[default] -- 22 =&amp;gt; 2222 (adapter 1)
[default] -- 8529 =&amp;gt; 5000 (adapter 1)
[default] Booting VM...
[default] Waiting for machine to boot. This may take a few minutes...
[default] Machine booted and ready!
[default] Mounting shared folders...
[default] -- /vagrant
&lt;/pre&gt;

&lt;p&gt;So, if our sandbox “node”/VM is running, we add the data repository capabilities.&lt;/p&gt;

&lt;h2 id=&quot;2-foxx-and-arangodb&quot;&gt;2. Foxx and ArangoDB&lt;/h2&gt;

&lt;p&gt;&lt;a href=&quot;https://www.arangodb.org/&quot;&gt;ArangoDB&lt;/a&gt; is a document-store built around the JavaScript V8 engine. ArangoDB supports schemaless, compressable data storage, as well as “actions”, a programmable data layer on top of the data store. These actions can again be programmed with &lt;a href=&quot;https://www.arangodb.org/foxx&quot;&gt;Foxx&lt;/a&gt;, a middleware on top of ArangoDB.&lt;/p&gt;

&lt;p&gt;To turn our fresh VM into a data repository, let’s login to the Vagrant box:&lt;/p&gt;

&lt;pre&gt;
 $ vagrant ssh
&lt;/pre&gt;

&lt;p&gt;Then we install ArangoDB. ArangoDB’s Ubuntu installation instructions can be found &lt;a href=&quot;http://www.arangodb.org/download&quot;&gt;here&lt;/a&gt; - in our case:&lt;/p&gt;

&lt;pre&gt;
sudo vi /etc/apt/sources.list.d/arangodb.list
&lt;/pre&gt;

&lt;p&gt;Add:&lt;/p&gt;

&lt;pre&gt;
 deb http://www.arangodb.org/repositories/arangodb/xUbuntu_12.04/ /
&lt;/pre&gt;

&lt;p&gt;Then, download the aptitude keys:&lt;/p&gt;

&lt;pre&gt;
wget http://www.arangodb.org/repositories/arangodb/xUbuntu_12.04/Release.key
&lt;/pre&gt;

&lt;p&gt;And:&lt;/p&gt;

&lt;pre&gt;
apt-key add - &amp;lt; Release.key
&lt;/pre&gt;

&lt;p&gt;And now, it’s easy, just:&lt;/p&gt;
&lt;pre&gt; 
  sudo apt-get update 
  sudo apt-get install arangodb=1.4.3
&lt;/pre&gt;

&lt;p&gt;Once the installation has finished, we have a basic ArangoDB setup running. This means, we have a document data store, and a RESTful middleware on top.&lt;/p&gt;

&lt;p&gt;So, let’s serve some movies.&lt;/p&gt;

&lt;h2 id=&quot;3-importing-movies&quot;&gt;3. Importing movies&lt;/h2&gt;

&lt;p&gt;Since movies and cinema give good case for data experiments, let’s fetch some data from the &lt;a href=&quot;http://www.grouplens.org/datasets/movielens/&quot;&gt;movielens free dataset&lt;/a&gt;. I leave this to an exercise to the reader, but maybe this &lt;a href=&quot;https://gist.github.com/mulderp/7796615&quot;&gt;gist&lt;/a&gt; will help you.&lt;/p&gt;

&lt;p&gt;So, once you have a JSON file of movies, we can easily import those into ArangoDB with:&lt;/p&gt;

&lt;pre&gt;
  arangoimp movies.json --type json --collection &quot;cinema_movies&quot;  --create-collection true
&lt;/pre&gt;

&lt;p&gt;In the Arango shell we now can test that movies are available with:&lt;/p&gt;

&lt;pre&gt;
  $ arangosh
  arangosh [_system]&amp;gt; db.cinema_movies.toArray()
  [
  {
    &quot;_id&quot; : &quot;movies/811811239&quot;,
    &quot;_rev&quot; : &quot;811811239&quot;,
    &quot;_key&quot; : &quot;811811239&quot;,
    &quot;year&quot; : 1997,
    &quot;title&quot; : &quot;Amistad&quot;,
    &quot;url&quot; : &quot;http://us.imdb.com/M/title-exact?imdb-title-118607&quot;
  },
  ...
&lt;/pre&gt;

&lt;p&gt;Next up, serving resources with Foxx.&lt;/p&gt;

&lt;h2 id=&quot;4-using-foxx&quot;&gt;4. Using Foxx&lt;/h2&gt;

&lt;p&gt;Foxx is all about building a smart dart layer on top of data. Foxx comes with the FoxxManager, a tool to manage HTTP for data storage on the node.&lt;/p&gt;

&lt;p&gt;So, let’s build a Foxx app:&lt;/p&gt;

&lt;pre&gt;
  $ mkdir cinema
  $ cd cinema
  $ vi manifest.json
&lt;/pre&gt;

&lt;p&gt;In the manifest, we manage the meta data of a Foxx application. Simply, it can look as:&lt;/p&gt;

&lt;pre&gt;
{
  &quot;name&quot;: &quot;cinema&quot;,
  &quot;version&quot;: &quot;0.0.1&quot;,
  &quot;author&quot;: &quot;patrick&quot;,
  &quot;controllers&quot;: {
    &quot;/&quot;: &quot;app.js&quot;
  }
}
&lt;/pre&gt;

&lt;p&gt;And some small JavaScript to serve the data:&lt;/p&gt;

&lt;pre&gt;
var Foxx = require(&quot;org/arangodb/foxx&quot;);
var app = new Foxx.Controller(applicationContext);
var _ = require(&apos;underscore&apos;);
var db = require(&quot;org/arangodb&quot;).db;

var Movie = Foxx.Model.extend({
  }, {
    attributes: {
      id: &quot;integer&quot;,
      url: &quot;string&quot;,
      title: &quot;integer&quot;,
      length: &quot;integer&quot;,
      genres: &quot;string&quot;
    }
});

&lt;/pre&gt;

&lt;p&gt;This is a bit similar to introducing a schema, e.g. as ElasticSearch does to help querying and storing data.&lt;/p&gt;

&lt;pre&gt;
app.get(&quot;/all&quot;, function(req, res) {
  res.json(_.map(db.cinema_movies.toArray(), function (movie) {
      return _.extend(new Movie(movie).attributes);
   }));
});
&lt;/pre&gt;

&lt;p&gt;We simply can load the application with the FoxxManager as follows:&lt;/p&gt;

&lt;pre&gt;
arangosh [_system]&amp;gt; fm.fetch(&quot;directory&quot;, &quot;/home/vagrant/cinema&quot;)
{
  &quot;path&quot; : &quot;/var/lib/arangodb-apps/databases/_system/cinema-0.0.1&quot;,
  &quot;app&quot; : &quot;app:cinema:0.0.1&quot;,
  &quot;code&quot; : 200
}
&lt;/pre&gt;

&lt;p&gt;Last, adding the dataset to the Foxx middleware with:&lt;/p&gt;

&lt;pre&gt;
arangosh [_system]&amp;gt; fm.mount(&quot;cinema&quot;, &quot;/home/vagrant/cinema&quot;)
{
  &quot;appId&quot; : &quot;app:cinema:0.0.1&quot;,
  &quot;mountId&quot; : &quot;39593354&quot;,
  &quot;mount&quot; : &quot;/home/vagrant/cinema&quot;,
  &quot;code&quot; : 200
}
&lt;/pre&gt;

&lt;p&gt;Now, our node acts as webserver and data source. Let’s try from the VM host:&lt;/p&gt;

&lt;pre&gt;
$ curl 0.0.0.0:5000/cinema/all
  [
  {
    &quot;year&quot; : 1997,
    &quot;title&quot; : &quot;Amistad&quot;,
    &quot;url&quot; : &quot;http://us.imdb.com/M/title-exact?imdb-title-118607&quot;
  },
  ...
&lt;/pre&gt;

&lt;p&gt;So, I think this is pretty great, since we can develop an API without running schema migrations, and directly work with data behind an API. With ArangoDB, we have a way to quickly provision a node with a web server and an application server, without going trough the manual installation of V8 and a database.&lt;/p&gt;

&lt;p&gt;Yet, in this approach are also quite some open questions: When to deploy your application from development to production? How to name databases and collections? How to apply the Foxx.Repository and Foxx.Model concepts best to serve the data?&lt;/p&gt;

&lt;p&gt;So, feedback would be great, and maybe you want to join the mailing list at: &lt;a href=&quot;https://groups.google.com/forum/#!forum/arangodb&quot;&gt;google groups&lt;/a&gt; - to discuss and learn together how to better drive single-page applications.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Tweaking Images</title>
   <link href="https://thinkingonthinking.com/tweaking-images/"/>
   <updated>2013-11-24T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/tweaking-images</id>
   <content type="html">&lt;p&gt;I am enjoying the browser more and more as a medium for presentations. In this post, I want to provide some tricks to transform images for presentations.&lt;/p&gt;

&lt;p&gt;First, I highly recommend browsing photographs at &lt;a href=&quot;http://flickr.com&quot;&gt;flickr.com&lt;/a&gt; when preparing a talk. Images capture emotions, as Garr Reynolds teaches us at &lt;a href=&quot;http://www.presentationzen.com/&quot;&gt;presentationzen.com&lt;/a&gt;. Once you found images, you can do this to combine them with &lt;a href=&quot;http://lab.hakim.se/reveal-js&quot;&gt;reveal.js&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;First, when you need to convert a JPG to PNG, you can do simply a:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;convert cinema.jpg cinema.png
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Often, you want to darken the images somewhat to use it as a background. For this, you can use:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;convert cinema.png -alpha on -channel A -evaluate set 40% +channel cinema_bg.png
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Sometimes, you also want to make a mashup or moodboard of images. This can be done with a trick from &lt;a href=&quot;http://gotofritz.net/blog/geekery/combining-images-imagemagick/&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;convert *.png[200x400] -background &quot;#ffffff&quot;  +append output.png
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Last, it can make sense to compress the image, in order to save valuable network bandwidth of readers. For this, I found a tool &lt;a href=&quot;http://pngquant.org/&quot;&gt;pngquant&lt;/a&gt;:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;pngquant --quality=65-80  cinema_bg.png
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;That’s all. Now, you can setup the image in your CSS with:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;.cinema_bg body {
    background-image: url(/static/images/cinema_bg-fs8.png);
    -webkit-background-size: cover;
    -moz-background-size: cover;
    -o-background-size: cover;
    background-size: cover;
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Hope this is helpful. Let me know what your favorite imagemagick tricks are!&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Thorax Lightning Talk - dotJS proposal</title>
   <link href="https://thinkingonthinking.com/thorax-talk/"/>
   <updated>2013-11-22T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/thorax-talk</id>
   <content type="html">&lt;p&gt;The DotJS talk proposal was rejected (as too framework specific), but I hold a talk on Thorax.js at the Munich Node Meetup:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Slides are &lt;a href=&quot;https://speakerdeck.com/mulderp/from-backbone-to-thorax&quot;&gt;here&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;Recordings are &lt;a href=&quot;http://www.youtube.com/watch?v=cH2zXBl6WXs&quot;&gt;here&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</content>
 </entry>
 
 <entry>
   <title>Push Conference - A Conference on Creativity and Flow</title>
   <link href="https://thinkingonthinking.com/Push-Conference/"/>
   <updated>2013-10-26T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/Push-Conference</id>
   <content type="html">&lt;p&gt;How amazing to experience a conference at the borders of design, art and technology. It was especially fascinating to merge with the flow of the speakers and the audience , who are driven by ideas on design and innovation. Here, 8 hours of talks feel like 10 seconds, and coming home, you feel full of energy to shape the world.&lt;/p&gt;

&lt;p&gt;One of the most inspiring messages were about play, discovery and learning. The speakers showed a lot of impressing examples of their works. However, maybe more interesting, were the many ways of how they failed.&lt;/p&gt;

&lt;p&gt;A notable talk came from the &lt;a href=&quot;http://www.field.io&quot;&gt;field design agency&lt;/a&gt;. At the beginning of the talk, the speaker mentioned that ideas from art, design and theory would be discussed. Interesting, because when you enter a certain daily routine in your life, words like “theory” or “art” are not often used. “Design” is used sometimes, but not so much in the sense of understanding and discovery, but more as an instrument to hide costs or emotions.&lt;/p&gt;

&lt;p&gt;The original goal of the talk by field.io is nicely summarized on the &lt;a href=&quot;http://push-conference.com/2013/speakers/field/&quot;&gt;conference website&lt;/a&gt; However, a lot of personal experiences came to light that happen when you work on something new and creative. And especially interesting, the speaker mentioned strategies to cope with failure and the outside perception of the creative process.&lt;/p&gt;

&lt;p&gt;Audiences of digital products almost never have a good feeling what they want. Therefore, a design process should be about inquiry. An important point was, that inquiry is not measured in weeks, but rather in months or years. That is why we need to have long-term goals: They are our personal frameworks to question the world and guide our ambitions.&lt;/p&gt;

&lt;p&gt;Having personal inquiries also implies that the same questions are asked over and over again. We will find new variations of existing answers. As a side-effect, we might find exactly the output that we get paid for as creative workers, as software developers or artists.&lt;/p&gt;

&lt;p&gt;Thanks, Markus Wendt, Mariana Santos, Wes Grubbs for amazing talks today, and the push conference team for an amazing conference!&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Learning from a Cinema Website</title>
   <link href="https://thinkingonthinking.com/Learning-From-Cinema-Websites/"/>
   <updated>2013-09-29T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/Learning-From-Cinema-Websites</id>
   <content type="html">&lt;p&gt;I just received a marketing flyer of the &lt;a href=&quot;http://www.nngroup.com/&quot;&gt;Nielsen Norman Group&lt;/a&gt;. It talks about design for small screens and usability of mobile applications (some background information is also shown &lt;a href=&quot;http://www.nngroup.com/articles/&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;

&lt;p&gt;Usability and low-bandwith mobile connections are important reasons to build interfaces with client-side frameworks such as Backbone.js. To show the potential for client-side applications and mobile interfaces, let’s do a small test with an existing cinema website. Where are problems today?&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;
    &lt;p&gt;First test: You want to chose a movie. The first thing you see are advertisements for movies you are not interested in. Resizing the window not even updates the contents to the essential content. Probably, the mobile version is not responsive either.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Second test: Navigation. Your goal is to see what movies are shown today. Oh, you need to hover with the mouse over “program” to see some further actions. The program overview is just far right in the sub-menu, and moving there require fine-motoric skills. Argh.. on a small screen, or on laptop with a trackpad, it is not fun.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Third test: Browsing. Nice, I am counting 20 movies, but how do I remove those movies about/with “Music”, “Animation” or “Fantasy”? At least, selecting a movie requires some trade-offs, like show-times, plots, actors and length. It is really difficult to do that comparison by scrolling the list of movies only. What I would prefer is some filter or sorting actions.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Well, looking at the source of the website, we see the sources for jQuery, jQuery-UI, and some libraries. In one library, we see a function that is tens of lines long:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;function filterProgram(tmpFilter, tmpField) { ... }
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Now compare the browsing experience, with the showcase &lt;a href=&quot;http://dpsgallery.adobe.com/#/main&quot;&gt;gallery demo from Adobe&lt;/a&gt;: Responsive, a nice feature project area, some search, and a filter for projects, all on one page.&lt;/p&gt;

&lt;p&gt;When applications become interfaces to support us with simple jobs (search/filtering/sorting), it is time to start learning client-side application development. And in my upcoming book on Backbone.js, I want to show an example of a better interface to browse movies (or products/news/…).&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Here goes your web stack</title>
   <link href="https://thinkingonthinking.com/Here-Goes-Your-Web-Stack/"/>
   <updated>2013-09-25T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/Here-Goes-Your-Web-Stack</id>
   <content type="html">&lt;p&gt;Where do the changes in web development come from? The main drivers come from users with small screens and/or mobile devices (users of iPads, iPads mini and iPhones). Here, we need to design interactions with screen regions, or explore document structures  that fit less easily into the relational data model. An excellent example on how new kinds of application stacks look like can be read &lt;a href=&quot;http://blog.fogcreek.com/the-trello-tech-stack/&quot;&gt;here&lt;/a&gt; on the technology stack at Trello.&lt;/p&gt;

&lt;p&gt;That was different some years ago and I remember the days when Apache and PHP were the main building blocks for web applications. In fact, there are still many requests for help with PHP on sites as Elance.com or on StackOverflow. PHP is still widely used.&lt;/p&gt;

&lt;p&gt;Yet, as easy it was (and is) to get started with PHP, applications in PHP are mainly about “dynamic” HTML. With emerging mobile devices, there are more requirements on applications for many users. For users, applications should be interactive and provide fast feedback as soon as they make an action.&lt;/p&gt;

&lt;p&gt;To make users happier, new layers of functionality are introduced in applications, like advanced frontend layers. We increasingly see people use tools as Ember.js, Marionette or Angular. To get this working and to combine interaction with features for recommendation and search, we need APIs. And there are interesting ideas to pull the routing layer into the database, such as &lt;a href=&quot;http://www.arangodb.org/2013/03/29/foxx-a-lightweight-javascript-application-framework-for-arangodb&quot;&gt;Foxx&lt;/a&gt; or &lt;a href=&quot;http://couchapp.org/page/index&quot;&gt;CouchApp&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Also, the technical blog by &lt;a href=&quot;http://artsy.github.io/&quot;&gt;Art.sy&lt;/a&gt; discusses many interesting questions from a web application that provides advanced interactions with content. And not for nothing, they often discuss the use of Backbone.js and MongoDB. Also the dev blog of &lt;a href=&quot;http://nerds.airbnb.com/&quot;&gt;Airbnb&lt;/a&gt; heavily shows the influence of mobile devices on application design. So, here goes your web stack …&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Start building libraries</title>
   <link href="https://thinkingonthinking.com/Software-Categories/"/>
   <updated>2013-09-18T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/Software-Categories</id>
   <content type="html">&lt;p&gt;Creating software feels similar to cooking. You need to select recipes, ingredients, and sometimes tools. And, mastering your gut feeling for becoming a software hero or Michelin-star cook takes time. A lot.&lt;/p&gt;

&lt;p&gt;On your path of learning what to do and don’t, you will need to make choices. Actually, a lot of choices. Cookbooks or frameworks help you choose to create menu’s or prepare a menu for a dinner party. In web development, Ruby-on-Rails, Twitter Bootstrap or Django help you to quickly satisfy a number of requirements, like dealing with dynamic content or mobile browsers.&lt;/p&gt;

&lt;p&gt;But before you jump from cookbook to cookbook, or from framework to framework, you sometimes will be better off by studying single recipe for a salad or a Lasagne. Similarly, single recipes are far more useful in the long term. A dinner party is only 3-5 times a year, while I need to search for simple cooking solutions almost every day. These simple setups are what we talk about when looking at Sinatra, Rack, Compass or Backbone.js. Whereas  Frameworks can bring high productivity, libraries give you tremendous levers in many different situations.&lt;/p&gt;

&lt;p&gt;Like in cooking, frameworks or menu’s can grow out of libraries or single recipes. As Ruby-on-Rails creator David Heinemeier Hansson explains in &lt;a href=&quot;http://bigthink.com/videos/big-think-interview-with-david-heinemeier-hansson&quot;&gt;this video&lt;/a&gt;: “Rails started as an application, it never started as [the framework] Rails. It started as me wanting to build Basecamp. … and I found a need to build a bunch of tools.”&lt;/p&gt;

&lt;p&gt;Yet where to draw the line between a menu (framework) and some recipe (library)? Or, as Andrzej asks in the blog post: &lt;a href=&quot;http://andrzejonsoftware.blogspot.de/2012/12/what-is-framework.html&quot;&gt;What is a framework?&lt;/a&gt; Menus (and framework) impress people, while endless practice with recipes (libraries) is the only way to become a star cook.&lt;/p&gt;

&lt;p&gt;Besides, applying a framework sooner or later confronts you with constraints. For example, with Twitter Bootstrap, how would you use parts of the responsive grid layout without using ‘span’ tags all across the page? Or with Rails, how would you develop a Rails application where you need an advanced database stack or interactive features?&lt;/p&gt;

&lt;p&gt;Although the choice of going for a framework or library can be vague, the end-goal for software is not. Software should be designed for others to use. As long as your users are facing standard problems, starting out with “frameworks” is fine.&lt;/p&gt;

&lt;p&gt;But from business strategy, it might be important to look at non-standard problems that only can be addressed by using libraries. Instead of managing ever increasing toolsets, it can be very profitable to build new kinds of libraries. Ask 37signals.&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Learning on Products</title>
   <link href="https://thinkingonthinking.com/Learnings-on-Products/"/>
   <updated>2013-08-14T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/Learnings-on-Products</id>
   <content type="html">&lt;p&gt;Lately, I was reflecting why I had studied engineering, and not art or design. (I had studied mathematics for some time, but switched backed to engineering. However, mathematics is already closer to design and art too.)&lt;/p&gt;

&lt;p&gt;My conclusion is that I enjoy the products of engineering: Mainly their functions, but in a sense the emotional rewards that certain products bring. In the case of electronic engineering, I mostly enjoyed music and sound processing devices. In general, I enjoyed communication devices, like radio or modems for communication via computers. The web revolutionized all communication devices, and fascination with products from Philips or Sony turned into fascination of products from Apple and Unix.&lt;/p&gt;

&lt;p&gt;Making good products requires some kind of philosophy or culture. It is difficult to say how it works, but for sure products just mirror the context where they were created. Therefore, good products involve &lt;a href=&quot;http://en.wikipedia.org/wiki/Design_thinking&quot;&gt;design thinking&lt;/a&gt;, looking at humans from different angles, and filtering certain aspects from our senses, how we feel or see things, or how we “experience” things.&lt;/p&gt;

&lt;p&gt;And here the circle closes: Culture and design are based on communication. Good products are based on communication. It is here where I want to try out something new: A newsfeed for sharing ideas on product design. Since I suspect most product designers today work in a startup environment, I want to dedicate the site Munich Technology Partners to startups too: &lt;a href=&quot;http://mtpartners.de&quot;&gt;mtpartners.de&lt;/a&gt;&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Event Driven Architecture</title>
   <link href="https://thinkingonthinking.com/event-driven-archicture/"/>
   <updated>2013-07-07T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/event-driven-archicture</id>
   <content type="html">&lt;p&gt;I recently looked at the source code of &lt;a href=&quot;https://github.com/antirez/lamernews&quot;&gt;Lamernews&lt;/a&gt;. It was very refreshing to see an application architecture that does not circulate around the MVC separation of concerns. Rather, the application design looks to circulate around ‘events’.&lt;/p&gt;

&lt;p&gt;Looking at the application, it is clear that the news domain is about communicating events. You can see this from some examples: &lt;a href=&quot;http://echojs.org&quot;&gt;Echo.js&lt;/a&gt; or &lt;a href=&quot;http://lamernews.com&quot;&gt;Lamernews&lt;/a&gt;. The main function is submitting news. Once news is in the newsfeed, news can be voted up or down and commented. The votes and comments influence the importance of news in the feed.&lt;/p&gt;

&lt;p&gt;When we would work with an MVC framework such as Ruby-on-Rails, we would probably start with mapping domain objects to routes. It would work nicely for CRUD actions on news. But for users that just browse through news, and vote news occasionally up or down, it is harder to see the usefulness of combining server-side actions with rendering logic and/or domain models for voting and commenting.&lt;/p&gt;

&lt;p&gt;This is where an event-driven architecture shines. And, some interesting points can be seen in the lamernews source:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;
    &lt;p&gt;All actions for working with news are accessible by API routes. For example, we see &lt;a href=&quot;https://github.com/antirez/lamernews/blob/master/app.rb#L819&quot;&gt;‘/api/votenews’&lt;/a&gt;  or ‘/api/postcomment’. These actions can directly be triggered from browser events with JavaScript as can be read from &lt;a href=&quot;https://github.com/antirez/lamernews/blob/master/public/js/app.js&quot;&gt;app.js&lt;/a&gt;. Also, the API responses work in a very lean and smart way. In most cases, a simple ‘status’ about processed events (ok/err) is enough.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;The ranking and scoring of news can be derived from aggregates of vote events. By using a smart recording of events, we get different ways to rank news (almost) for free. Smart operations on aggregates is also where Redis shines. When a vote event occurs, we increase a counter for the vote in the news hash, recalculate the rank of the news, and store this in the sorted set ‘news.top’ (see &lt;a href=&quot;https://github.com/antirez/lamernews/blob/master/app.rb#L1431-L1443&quot;&gt;compute_news_score&lt;/a&gt;). Next time, we serve news, we just need to look up the top news in the sorted set.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Now, what can we take away from this? For projects such as social communities or Ajax heavy applications, we might benefit from making events accessible from an API. We also might see cases, such as lamernews shows, where we can skip the ORM setup and profit from ‘smart writes’ to hashes and sorted sets. Well, these types of lean API backends combined with a smart noSQL storage are also great for prototyping responsive interfaces, as I experienced during the last weeks during writing my book on Backbone.js.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Writing On Ruby and Backbone.js</title>
   <link href="https://thinkingonthinking.com/writing-on-ruby-and-backbone/"/>
   <updated>2013-06-07T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/writing-on-ruby-and-backbone</id>
   <content type="html">&lt;p&gt;Writing is about composition, argumentation and ideas. This makes writing very similar to programming.&lt;/p&gt;

&lt;p&gt;Not suprisingly, David Heinemeier Hansson, main author of the Ruby-on-Rails framework, mentioned &lt;a href=&quot;http://37signals.com/svn/posts/3375-the-five-programming-books-that-meant-most-to-me&quot;&gt;books about writing&lt;/a&gt; as path to become a better programmer. What we as writers discover is: Style is hard, finding the right words, applying grammatical rules and sharing preferences.&lt;/p&gt;

&lt;p&gt;This is what I am practicing the last months on a book idea. I want to show how small server-side and client-side components can be combined for better interactions with users. The book is about combining ideas from Ruby and JavaScript.&lt;/p&gt;

&lt;p&gt;Ruby has been a great tool in my programming expeditions. Ruby is about a programmer’s happiness. Ruby provides some of the cleanest and most elegant syntax for a programming language I know of. Syntax is crucial. Ruby allows to form expressions that are very close to the line of thought we do in everyday thinking:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; Journey.new.visit(barcelona).take_photographs([segrada_familia, beach]) 
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Recently, I feel that Backbone.js provides an interesting extension to this line of thinking. In contrast to Ruby, however, Backbone.js communicates differently with a programmer. Backbone.js provides hooks for ‘listening’ to events, and ‘rendering’  and ‘transport’ of data. So, the set of verbs for in Ruby and Backbone is similar and different (think on the verbs: save, find, render, each, map, rescue). However, the similar and different semantics are very relevant to the server or browser environments where the verbs are used.&lt;/p&gt;

&lt;p&gt;Additionally, Ruby and Backbone (with Underscore.js) provide nice ways to compose objects, and encapsulate processes and data. Whereas in Ruby-on-Rails, we intuitively can work with storage and HTTP, Backbone.js provides Views, Events, and (also) Collections and Models. The similarities and differences among server and client will be the topic of my book**, that I hope to publish after the summer. My blog updates will be less frequent therefore.&lt;/p&gt;

&lt;p&gt;** It is the topic of the talk I will be giving at &lt;a href=&quot;http://www.rubyday.it/schedule&quot;&gt;Rubyday.it 2013 in Milano&lt;/a&gt;&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Preparing Application Development</title>
   <link href="https://thinkingonthinking.com/Preparing-Application-Development/"/>
   <updated>2013-05-21T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/Preparing-Application-Development</id>
   <content type="html">&lt;p&gt;I recently discovered a github repo by &lt;a href=&quot;https://twitter.com/maccaw&quot;&gt;@maccaw&lt;/a&gt; that looked very interesting: &lt;a href=&quot;https://github.com/maccman/catapult&quot;&gt;Catapult&lt;/a&gt;. For building prototypes of some client-side code, or as frontend asset server based on Ruby, Catapult gives you all you need. With:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;catapult new &amp;lt;squash&amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;You get some fresh project structure, an index.html, a small server and some basic rake tasks to connect with a production environment. This is a bit similar to what Yeoman does, but the advantage of Catapult (for me) is that it is Ruby.&lt;/p&gt;

&lt;p&gt;Now, some savings in typing a few ‘mkdir -p /public/js/libs’, copying an index.html, and providing a shortcut for running a Rack server might sound not as a big deal. However, it sets a nice background for something great: Managing application dependencies, and especially, client-side application dependencies.&lt;/p&gt;

&lt;p&gt;When you start with a client-side application from scratch, you need to get jQuery, Underscore.js, maybe Backbone.js, Marionette.js, Handelbars.js, and some other library here and there, and it is easy to lose the overview where to get your libs, and what version to import. In Ruby land, you only need to say ‘bundle install’ and you are ready to go.&lt;/p&gt;

&lt;p&gt;It is here where I realized (within a discussion at Github issues) what another very interesting tool is about: &lt;a href=&quot;https://github.com/bower/bower&quot;&gt;‘bower’&lt;/a&gt;. Bower is a Node.js based dependency management solution. It reads a definition of dependencies from a bower.json file, and when you run ‘bower install’ your dependencies are fetched. It’s basically inserting the DNA into an empty application: Cloning the project’s into your local machine, and copy it from there into the application.&lt;/p&gt;

&lt;p&gt;It’s here where I am currently having fun. And it is here where &lt;a href=&quot;https://github.com/yeoman&quot;&gt;Yeoman&lt;/a&gt; seems to provide even better ways to kickstart a new application. However, as a Ruby developer, Bower and Yeoman easily introduce a high dose of JavaScript, which might distract from your domain-driven design approaches and RESTful API’s. Also, having the whole history of your dependencies in your prototyping sandbox is not always handy. I prefer to actually get the actual JavaScript sources that I need, and therefore, I need to use bower-installer. Maybe bower-ruby or bower-rails are good to go with too. It will need some exploration still.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>NoSQL Matters 2013 - A conference on scaling data</title>
   <link href="https://thinkingonthinking.com/no-sql-matters-conference-on-scaling-data/"/>
   <updated>2013-04-27T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/no-sql-matters-conference-on-scaling-data</id>
   <content type="html">&lt;p&gt;Not sure whether you should be interested in NoSQL? Well, I was not sure either, and that was an important reason to visit the &lt;a href=&quot;http://2013.nosql-matters.org/cgn/&quot;&gt;NoSQL Matters 2013&lt;/a&gt; in Cologne. If you reflect, it is data that occupies a lot of our working life as software developers, and, technically, there are a lot of hidden assumptions made, when touching data questions.&lt;/p&gt;

&lt;p&gt;First, there are assumptions around structuring data. Machines seem to assume that we should structure our data as much as possible, providing at least some tables (or schemas) upon which algorithms can easily operate. For engineers, that would not be too big of a problem. However, we as humans in general (and business people in particular), often encounter confusions. What level of ambiguity can we allow when we are doodling data models? What if your business problem tomorrow looks different from our design today? How to migrate data between different systems, while not impacting a running business?&lt;/p&gt;

&lt;p&gt;It is here, where a visit to the NoSQL country can help you see new options. Key-value stores and document-oriented database simplify storing data for prototyping or distribution, while graph databases provide almost unlimited ways to look for new patterns in data. Basically, with a NoSQL mindset we are better able to cross boundaries between persistence, distribution and querying of data and - like applying different filters - NoSQL solutions can help to separate signals from noise.&lt;/p&gt;

&lt;p&gt;Second, the cost of data and data processing is still decreasing, maybe slower, but the trend is still downwards. And as often happened in computing, this will give rise to some wave of disruptive innovation. For tackling data persistence and distribution problems, I think this is where using open-source software will become much more viable than buying into ‘luxury’ database products from software vendors. This makes a visit to a NoSQL conference also very interesting, since many open-source developers are looking for feedback and solutions to their problems. And, projects on e.g. Github provide a much better platform to discuss concepts and shortcomings of dealing with data, than buying into todays (and tomorrows) promisses of existing database vendors.&lt;/p&gt;

&lt;p&gt;Third, at a NoSQL conference, you feel that wrapping data with services helps to hide a lot of nitty-gritty details from using the data in applications. Similarly, as renting a car that brings you from A to B, an API is just enough abstraction to hide technical concern today from business requirements tomorrow. As such, APIs provide some common ground for designing the architecture of distributed systems and communicating transactions across a network.&lt;/p&gt;

&lt;p&gt;All in all, NoSQL Matters 2013 in Cologne was a very rewarding conference. As many NoSQL developers will stress, SQL will not lose its relevance for sure, it’s a good standard for dealing with some classes of problems, but at least for me, key-value stores, documents, vector clocks and graphs start slowly to make a lot more sense.&lt;/p&gt;

&lt;h1 id=&quot;some-references&quot;&gt;Some references&lt;/h1&gt;

&lt;ul&gt;
  &lt;li&gt;Learning to think with distributed data structures @coderoshi - &lt;a href=&quot;https://github.com/coderoshi/dds&quot;&gt;https://github.com/coderoshi/dds&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;Martin Fowlers’ Aggregation pattern - &lt;a href=&quot;http://martinfowler.com/bliki/AggregateOrientedDatabase.html&quot;&gt;http://martinfowler.com/bliki/AggregateOrientedDatabase.html&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;Query data with Graphs - &lt;a href=&quot;http://de.slideshare.net/jexp/intro-to-neo4j-presentation&quot;&gt;http://de.slideshare.net/jexp/intro-to-neo4j-presentation&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;Redis = (Remote) Data Structure Server - &lt;a href=&quot;https://speakerdeck.com/czarneckid&quot;&gt;https://speakerdeck.com/czarneckid&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;Foxx - prototyping APIs with ArangoDB - &lt;a href=&quot;https://speakerdeck.com/moonglum/arangodb-foxx&quot;&gt;https://speakerdeck.com/moonglum/arangodb-foxx&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</content>
 </entry>
 
 <entry>
   <title>On designing interfaces - Learning from patterns</title>
   <link href="https://thinkingonthinking.com/ruby-and-patterns/"/>
   <updated>2013-04-05T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/ruby-and-patterns</id>
   <content type="html">&lt;p&gt;Interfaces are important in programming, as in business or relationships. By looking at Ruby-on-Rails, a web framework that is widely known for its implementation of the Model-View-Controller pattern, a lot can be discovered on interface design.&lt;/p&gt;

&lt;p&gt;When we first touch &lt;a href=&quot;http://martinfowler.com/eaaCatalog/&quot;&gt;architecture design patterns&lt;/a&gt;, we can feel quickly overwhelmed by ‘theoretical’ or very ‘abstract’ discussions. But after many hours of software development, these patterns start jumping into your eyes suddenly. And then, patterns can act as &lt;a href=&quot;http://en.wikipedia.org/wiki/Loophole&quot;&gt;loophole&lt;/a&gt; between achieving perfect design in theory, and avoiding a chaos design in practice.&lt;/p&gt;

&lt;p&gt;So, to understand how design patterns can influence good interface design, we can look at the Rails’ directory structure:&lt;/p&gt;

&lt;pre&gt;
      ├── actionpack
      │   ├── lib
      │   │   ├── action_controller
      │   │   ├── action_view
      │   └── test
      ├── activerecord
      │   ├── lib
      │   └── test
&lt;/pre&gt;

&lt;p&gt;Looking at this (filtering the ‘tree -L 2’ from the Rails repository helps), the following similarities between &lt;a href=&quot;http://martinfowler.com/eaaCatalog/&quot;&gt;Martin Fowler’s Enterprise Patterns&lt;/a&gt; can be seen:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;http://martinfowler.com/eaaCatalog/pageController.html&quot;&gt;Page Controller&lt;/a&gt;/&lt;a href=&quot;http://martinfowler.com/eaaCatalog/frontController.html&quot;&gt;Front Controller&lt;/a&gt;: The Ruby-on-Rails MVC stack handles requests from a Router, that are then processed by an instance of ActionController. Comparing to Fowler’s pattern definitions, a Router can be interpreted as “handling all requests for a Web site” and an ActionController instance as “handling requests for a specific pages or actions”. This design works very well to create, update or render resources, and map actions to URLs in a RESTful manner. Furthermore, Rails developers will know very well how common logic for e.g. authentication can be extracted into a &lt;a href=&quot;http://martinfowler.com/eaaCatalog/applicationController.html&quot;&gt;Application Controller&lt;/a&gt;&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;http://martinfowler.com/eaaCatalog/templateView.html&quot;&gt;Template View&lt;/a&gt;: According to Fowler, this pattern is about “rendering information into HTML by embedding markers in an HTML page”. As such, it describes the purposes of views in Ruby-on-Rails. Interestingly, Fowler describes some alternative options, such as &lt;a href=&quot;http://martinfowler.com/eaaCatalog/transformView.html&quot;&gt;Transform View&lt;/a&gt;, or &lt;a href=&quot;http://martinfowler.com/eaaCatalog/twoStepView.html&quot;&gt;2-step View&lt;/a&gt;. This shows that view templates has its limits, and is not the best solution for all rendering problems. Maybe in a later post, we can look at how client-side MVC framework apply different rendering approaches.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;&lt;a href=&quot;http://martinfowler.com/eaaCatalog/activeRecord.html&quot;&gt;ActiveRecord&lt;/a&gt;: ActiveRecord is a so-called data source architectural pattern, and is about “encapsulating database access, and adding domain logic on that data”. It is one of the patterns besides MVC that helped Rails into becoming what is today. For non-experts, ActiveRecord helps to encapsulate e.g. a Person data record. An instance of a Person that is derived from ActiveRecord, gives a nice interface to manage ‘persistence’ of the Person state, as well as ways to manage the context of a person (such as associations, validations, custom extensions). Thanks to &lt;a href=&quot;https://twitter.com/vicentereig&quot;&gt;@vicentereig&lt;/a&gt; for pointing me to a post by &lt;a href=&quot;http://blog.codeclimate.com/blog/2012/10/17/7-ways-to-decompose-fat-activerecord-models/&quot;&gt;@bryanry&lt;/a&gt; on keeping the ActiveRecord pattern under control.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;There are probably a dozen more interesting patterns to discover in the Rails repository. ActiveModel is somewhat related to deal with aggregation oriented databases, the adapter pattern (e.g. for ORMs) is related to this as well. Furthermore, you can see usages of Observers (e.g. in the ActiveSupport library) and Builders (e.g. for generators).&lt;/p&gt;

&lt;h1 id=&quot;conclusions&quot;&gt;Conclusions&lt;/h1&gt;

&lt;p&gt;Patterns feel somewhat ‘theoretical’ when first touched. But as theory is to engineering, design patterns can guide you in making good, practical design decisions. Patterns can be used for communication, or to identify interfaces between components. Patterns appear at later stage of software projects, and are as such less helpful for spiking or prototyping some ideas in early software design projects.&lt;/p&gt;

&lt;h1 id=&quot;references&quot;&gt;References&lt;/h1&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;The book ‘Enterprise Architecture’ by Martin Fowler and ‘Domain Driven Design’ have a lot of interesting examples how patterns can lead to interface and vice-versa.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;A topic that is closely related to pattern and interface design is refactoring. Also here, Martin Fowler keeps track of a catalog: &lt;a href=&quot;http://www.refactoring.com/catalog/index.html&quot;&gt;Refactoring.com&lt;/a&gt;. I am grateful to &lt;a href=&quot;https://twitter.com/rstankov&quot;&gt;@rstankov&lt;/a&gt; for sharing a great refactoring session, that showed me how removing code can improve interfaces (and as such promote pattern discovery).&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Apart from Rails, some nice interfaces in Ruby projects can be found in e.g. &lt;a href=&quot;https://github.com/plataformatec/devise/tree/master/lib/devise/orm&quot;&gt;Devise’s ORM adapters&lt;/a&gt; or &lt;a href=&quot;https://github.com/lostisland/faraday/blob/master/lib/faraday/connection.rb#L28&quot;&gt;Faraday’s connection builder&lt;/a&gt;.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;
</content>
 </entry>
 
 <entry>
   <title>Writing a book - a lesson in pitching</title>
   <link href="https://thinkingonthinking.com/writing-a-book/"/>
   <updated>2013-03-29T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/writing-a-book</id>
   <content type="html">&lt;p&gt;I was recently approached by some smaller technical publisher with the idea of writing a book on some things I also write here. As I like writing, and I have been exploring web ideas for publishing some time ago, I was very intrigued to look more into this. An idea for the draft outline for the book came from the publisher, and it looked like the default outline of most computer books: Starting with ‘hello world’, discuss some testing, and round up with some practical use cases.&lt;/p&gt;

&lt;p&gt;I asked myself: Why would I choose the publisher who approached me, and why would I not try with one of the well known publishers for dynamic programming languages and/or web programming? After all, if I invest time in research and a manuscript draft, I might even aim at getting access to great editors, distribution channels and well tested tools for publishing. Or, I reflected, why does it make sense to choose a publisher at all, and not go for self-publishing.&lt;/p&gt;

&lt;p&gt;So, with some different ways for publishing in the back of my mind, I started to research material for a draft book proposal and a sample chapter. It felt a bit like programming, you explore ideas, develop slowly a structure and try to get the basic names and concepts right. And, as often happens with programming too, it is easy to quickly get lost in unnecessary details. So, although I quickly made progress with the draft, I stopped asking myself the important question: Whom am I pitching this book proposal to?&lt;/p&gt;

&lt;p&gt;After all, there are a number of different publishers out there, that have a different focus. It is like there a publishers that are specialized in gardening, cooking or sports. So far so good, but if you now pitch as a golf player to a publisher for gardening, chances are high that your pitch will fail, although both probably know a lot about lawns.&lt;/p&gt;

&lt;p&gt;As I think that feedback for learning is very important, I pitched my draft proposal to a publisher some days ago. This was my first contact with a publisher from the perspective of an author. And, the pitch failed. However, this is a good lesson to learn. It is a lesson on pitching yourself, instead of pitching ideas from others: Book projects happen in the context of larger book themes at publishers. It is the understanding of these themes, that make you as a reader choose to buy a book from a certain publisher. And, it is the intuition of publishers to choose authors that fit their ideas and themes. So, although I am probably going to post-pone my book project for a while (since it is a huge investment in time and effort), I am going to look closer now at the books that I read, and why a publisher might publish them. After all, books can help us to share what can be discovered, or like &lt;a href=&quot;http://en.wikipedia.org/wiki/Martin_Heidegger&quot;&gt;Martin Heidegger&lt;/a&gt; said: “Tell me what you read and I’ll tell you who you are.”&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Fashion and Business models</title>
   <link href="https://thinkingonthinking.com/fashion-and-business-models/"/>
   <updated>2013-03-08T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/fashion-and-business-models</id>
   <content type="html">&lt;p&gt;Fashion is about building &lt;em&gt;remarkable&lt;/em&gt; things.&lt;/p&gt;

&lt;p&gt;Making a remarkable product does not necessarily require (much) money or (much) time. For example, designer and software developer Sacha Greif says starting a &lt;a href=&quot;http://sachagreif.com/the-side-project-project/&quot;&gt;sideproject&lt;/a&gt; only requires 10 hours.&lt;/p&gt;

&lt;p&gt;How do fashion designers build products that feel special? How can you connect with people and address their needs?&lt;/p&gt;

&lt;p&gt;Seth Godin explains exactly this in a TED talk: &lt;a href=&quot;http://www.ted.com/talks/seth_godin_on_sliced_bread.html&quot;&gt;How to get your ideas spread&lt;/a&gt;&lt;/p&gt;

&lt;iframe src=&quot;https://embed.ted.com/talks/seth_godin_on_sliced_bread&quot; width=&quot;640&quot; height=&quot;360&quot; frameborder=&quot;0&quot; scrolling=&quot;no&quot; webkitallowfullscreen=&quot;&quot; mozallowfullscreen=&quot;&quot; allowfullscreen=&quot;&quot;&gt;&lt;/iframe&gt;

&lt;p&gt;In this video, Godin looks for explanations on how Jeff Koons or Frank Gehry became so successful. He also mentions, that sliced bread (that had nice patents and a well-thought out bakery process by its inventor) got unnoticed for 15 years, and became only successful by having a company applying the right marketing techniques. His message is about looking at fashion, because fashion is all about making products ‘remarkable’.&lt;/p&gt;

&lt;p&gt;In the &lt;a href=&quot;http://bmfiddle.com/&quot;&gt;business model canvas&lt;/a&gt;, selling a product or service is about the customer relationship and sales channel part. It’s about how you feel, when you enter and touch a shop or a product. It’s providing the right options to buy a product, or subscribe to a service. It’s about the setup of a flagship store or the design of a landing page.&lt;/p&gt;

&lt;p&gt;In the world of fashion, the most remarkable products are “haute couture”. These fashion creations provide ideas to a whole industrial branch. The ideas that are shown on runways easily spread through magazines, television shows and blogs nowadays.&lt;/p&gt;

&lt;p&gt;In another sense, garments from designers such as Jean Paul Gaultier are also a kind of prototype. A prototype (greek for “first impression”) is about how something can look, work, feel and how you can appreciate it . The saying goes, a picture is worth a thousands words, and a prototype is worth a thousand pictures. In the fashion pyramid, prototyping is about what happens at the top of the pyramid. There, fashion designers are specialized in sketching, observing and in handcrafting and materializing ideas.&lt;/p&gt;

&lt;p&gt;In business models, prototyping is about something similar: Discovering unique and special recipes from sometimes common ingredients, sometimes rare and expensive ones (like patents or jewels). And in both, business models and fashion, it’s important to learn what works for 1 or 2 customers, for 100 men or women, for a cultural region, or for a contintent. (Another example: Some companies seem to run prototypes of a business in Canada first, and roll them out in the USA afterwards)&lt;/p&gt;

&lt;h2 id=&quot;conclusions&quot;&gt;Conclusions&lt;/h2&gt;
&lt;p&gt;Here are some conclusions:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;Business models don’t need a lot of resources to experiment with. That’s the whole point of building prototypes. Fast, cheap and throw-away models to test ideas.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;A business prototype might be compared to ‘haute couture’ in the fashion pyramid. It’s about making something unique and special, that can be simplified later on. Like in haute couture, there are companies specializing in bringing prototypes to the world; however, a whole lot profitable luxury fashion lines do both: Haute-couture and High-end retail. And besides, there is space for volume and mass market segments.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;A business model is not just a product and a customer. Like in fashion, it’s about how you feel in a shop, are received by a nice sales person, get extras and feel special. That is the sales channel and customer relationship part in a business model.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Note: This post was inspired by a &lt;a href=&quot;https://groups.google.com/forum/?hl=de&amp;amp;fromgroups=#!topic/lean-startup-circle/F2bk3MuOZDQ&quot;&gt;a discussion in the lean startup circle&lt;/a&gt; on how small companies can actually have enough resources to experiment with a business model.&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Building a flexible Rails Stack with Chef</title>
   <link href="https://thinkingonthinking.com/A-custom-rails-stack-with-chef/"/>
   <updated>2013-02-27T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/A-custom-rails-stack-with-chef</id>
   <content type="html">&lt;p&gt;&lt;b&gt;Edit&lt;/b&gt;: If you are looking for the best experience to setup infrastructure for Ruby-on-Rails, I would suggest to look at &lt;a href=&quot;http://twitter.com/mlafeldt&quot;&gt;@mlafeldt&lt;/a&gt;’s cookbook for &lt;a href=&quot;https://github.com/elm-city-craftworks/practicing-ruby-cookbook&quot;&gt;practicing-ruby&lt;/a&gt;. The accompanying article is great too: &lt;a href=&quot;http://tinyurl.com/pr-chef&quot;&gt;Infrastructure automation by example&lt;/a&gt;. If you are looking to understand more on cookbooks for wrapping Ruby with Chef, some more details can be found in &lt;a href=&quot;http://thinkingonthinking.com/Chef-and-Rubies/&quot;&gt;this post&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Most of us who develop Ruby based web applications will know that the setup and configuration of a stack of web infrastructure can be tricky:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;building Rubies including Gemsets often require system libraries that are different on Debian 64-Bit, Fedora, …&lt;/li&gt;
  &lt;li&gt;setup of webservers (nginx or apache) have special compile flags and configuration settings too&lt;/li&gt;
  &lt;li&gt;adaptors for Rack (Passenger or Unicorn nowadays) need to be built or wrapped in scripts&lt;/li&gt;
  &lt;li&gt;databases are evolving (RDBMS, document-stores, key-value stores, graph db’s … ) and often come in pairs&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Additionally, your Unix most likely will need some essential build packages such as compilers, git, image processors, messaging queues, mailservers, etc, as well as Unix users and groups for basic security settings. Last but not least, there are different Unix distributions which bring additional surprises.&lt;/p&gt;

&lt;p&gt;So, even when opinions in Rails keep some decisions simple, a web development stack might differ from the one that you or your customer will use in production.&lt;/p&gt;

&lt;p&gt;With all these choices, how can you test opinions on a web stack that are good for you?&lt;/p&gt;

&lt;p&gt;Particulary, how can you test opinions in the context of a database-driven web application such as Ruby-on-Rails?&lt;/p&gt;

&lt;p&gt;I try to answer this with the help of 2 tools: &lt;a href=&quot;http://docs.vagrantup.com/v1/docs/provisioners/chef_solo.html&quot;&gt;Chef-solo&lt;/a&gt; and &lt;a href=&quot;http://berkshelf.com/&quot;&gt;Berkshelf&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;1-abstractions-in-chef&quot;&gt;1. Abstractions in Chef&lt;/h2&gt;

&lt;p&gt;What Rails is to a database-driven web application, Chef might be to infrastructure: Opinions in code. But as far as I can judge, the ecosystem of infrastructure is much more diverse, than the different components of a model-view-controller design. Let’s see how Chef attempts to abstract away the diversity in infrastructure:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;One of the most important abstractions in Chef are cookbooks. Cookbooks are similar to Gems in Ruby. Cookbooks bundle code (= “recipes”) that can be configured with “attributes” and configuration “templates”. In a way, cookbooks apply the MVC pattern: Data models are attributes, views are templates, and controllers are recipes. Cookbooks are opinionated, some more than others, but my advice for installation of services, is to look at &lt;a href=&quot;http://community.opscode.com&quot;&gt;community cookbooks&lt;/a&gt; first, and then at opinions found in cookbooks found at Github.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;As we just tackle a single node setup, Vagrant and Chef-Solo are a sweet combination. Vagrant takes care of your Unix VM, and Chef-Solo can easily load cookbooks from your local environment.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;The setup of a web infrastructure stack will require multiple cookbooks, and you’ll quickly feel that downloading cookbooks yourself, and variation of cookbooks should be managed with some tool. So far, I tried 2 tools to manage cookbooks: &lt;a href=&quot;http://berkshelf.com/&quot;&gt;Berkshelf&lt;/a&gt; and &lt;a href=&quot;https://github.com/applicationsonline/librarian&quot;&gt;Librarian-Chef&lt;/a&gt;. Whereas Librarian manages quite well the download part, Berkshelf helps in solving package problems at hand. Think of Berkshelf as the Bundler for Ruby gems and in let’s say 80% of cases, you’ll just want to download cookbooks and can live with cookbook defaults, rather than deriving your own version of a cookbook. Berkshelf helps to abstract away the details when needed.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Vagrant is your infrastructure lab. I wrote about some first steps with Vagrant, &lt;a href=&quot;http://thinkingonthinking.com/An-experiment-with-Vagrant-and-Neo4J/&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;http://thinkingonthinking.com/building-a-Vagrant-base-box/&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;http://thinkingonthinking.com/minimum-nginx-node-with-librarian-chef/&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;an-overview-of-a-rails-stack&quot;&gt;An overview of a Rails stack&lt;/h2&gt;

&lt;p&gt;I hope the abstractions above can give you some taste of what is coming up in this section. First, let’s think on a Rails application stack:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;RVM to manage Rubies (could be rbenv too)&lt;/li&gt;
  &lt;li&gt;Passenger to consume and respond with requests for/from Rack (could be Unicorn)&lt;/li&gt;
  &lt;li&gt;PostgreSQL to deal with data storage&lt;/li&gt;
  &lt;li&gt;Nginx as proxy for logging&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In Chef language, you would want to look at application cookbooks, such as &lt;a href=&quot;http://community.opscode.com/cookbooks/application_ruby&quot;&gt;application_ruby&lt;/a&gt; or &lt;a href=&quot;http://community.opscode.com/cookbooks/passenger_apache2&quot;&gt;passenger_apache2&lt;/a&gt;. Nathen Harvey shows how to get going with the passenger_apache2 cookbook in his &lt;a href=&quot;http://www.nathenharvey.com/blog/2012/12/07/learning-chef-part-2/&quot;&gt;blog&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;But for fun and for learning let’s look into the setup of a custom stack.&lt;/p&gt;

&lt;p&gt;The dependencies for above’s stack could be defined with &lt;a href=&quot;http://berkshelf.com/&quot;&gt;Berkshelf&lt;/a&gt; in the Berksfile as follows:&lt;/p&gt;

&lt;pre&gt;
site :opscode

cookbook &apos;apt&apos;
cookbook &apos;build-essential&apos;
cookbook &apos;users&apos;
cookbook &apos;rvm&apos;, :git =&amp;gt; &quot;https://github.com/fnichol/chef-rvm&quot;
cookbook &apos;nginx&apos;
cookbook &apos;postgresql&apos;
&lt;/pre&gt;

&lt;p&gt;Running a:&lt;/p&gt;

&lt;pre&gt;
  berks install
&lt;/pre&gt;

&lt;p&gt;imports the cookbooks to your local Chef environment. Next, we need to setup the Vagrantfile to use Chef-solo provisioning:&lt;/p&gt;

&lt;pre&gt;
require &apos;multi_json&apos;
require &apos;berkshelf/vagrant&apos;

Vagrant::Config.run do |config|
  VAGRANT_JSON = MultiJson.load(Pathname(__FILE__).dirname.join(&apos;.&apos;, &apos;node.json&apos;).read)

  config.vm.provision :chef_solo do |chef|
    chef.cookbooks_path = [&quot;cookbooks&quot;]
    chef.json = VAGRANT_JSON
    chef.log_level = :debug
  
    VAGRANT_JSON[&apos;run_list&apos;].each do |recipe|
      chef.add_recipe(recipe)
    end if VAGRANT_JSON[&apos;run_list&apos;]
  end
end
&lt;/pre&gt;

&lt;p&gt;Ha, there is node.json loaded, and that is Chef’s way to define a node setup. Looking more carefully above, there are 2 details: The “run_list” and the “chef.json” part. The run_list defines, how a node conversion flows, while the json overrides attributes from cookbooks.&lt;/p&gt;

&lt;p&gt;The setup of these can be tricky:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Refer to &lt;a href=&quot;http://thinkingonthinking.com/2013-12-11-Chef-and-Rubies&quot;&gt;this post&lt;/a&gt; on how get Chef rwapping Ruby. Right now, on default VMs in Linux distrubtions, there is a an old Ruby 1.8.7. Those need an upgrade.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;passenger&quot;&gt;Passenger&lt;/h2&gt;

&lt;p&gt;Passenger is a Ruby gem that is partly compiled into Nginx or loaded as module in Apache. Let’s look at the nginx setup. The tricky parts are the system paths, thanks a lot to &lt;a href=&quot;https://twitter.com/jtimberman&quot;&gt;@jtimberman&lt;/a&gt; for helping me understand this better. There are interesting details behind this in yesterday’s &lt;a href=&quot;http://www.youtube.com/watch?v=ddMLvMvOUfg&amp;amp;feature=youtu.be&quot;&gt;office hours with Joshua&lt;/a&gt;.&lt;/p&gt;

&lt;pre&gt;
  ...
  &quot;nginx&quot;: {
    &quot;version&quot;: &quot;1.2.5&quot;,
    &quot;user&quot;: &quot;deploy&quot;,
    &quot;init_style&quot;: &quot;init&quot;,
    &quot;modules&quot;: [
      &quot;http_stub_status_module&quot;,
      &quot;http_ssl_module&quot;,
      &quot;http_gzip_static_module&quot;
    ],
    &quot;passenger&quot;: {
      &quot;version&quot;: &quot;3.0.18&quot;,
      &quot;gem_binary&quot;: &quot;/usr/local/rvm/wrappers/ruby-2.0.0-p0/gem&quot;
    },
    &quot;configure_flags&quot;: [
      &quot;--add-module=/usr/local/rvm/gems/ruby-2.0.0-p0/gems/passenger-3.0.18/ext/nginx&quot;
    ],
  ...
&lt;/pre&gt;

&lt;ul&gt;
  &lt;li&gt;Add some gems that you mostly will need: Rake, Bundler and Rails:&lt;/li&gt;
&lt;/ul&gt;

&lt;pre&gt;
  ...
  &quot;rvm&quot;: {
    &quot;rubies&quot;: [&quot;2.0.0-p0&quot;],
    &quot;default_ruby&quot;: &quot;2.0.0-p0&quot;,
    &quot;vagrant&quot;: { &quot;system_chef_solo&quot; : &quot;/opt/chef/bin/chef-solo&quot; },
    &quot;gems&quot;: {
      &quot;2.0.0-p0&quot;: [
        {&quot;name&quot;: &quot;bundler&quot;},
        {&quot;name&quot;: &quot;rake&quot;},
        {&quot;name&quot;: &quot;rails&quot;, &quot;version&quot;: &quot;4.0.0.beta1&quot; }
      ]
    }
  ...
&lt;/pre&gt;

&lt;p&gt;Maybe there are more details, but maybe you are curious to just see the &lt;a href=&quot;https://github.com/mulderp/chef-rails-stack/tree/rails4_stack&quot;&gt;chef-rails-stack&lt;/a&gt; at work.&lt;/p&gt;

&lt;p&gt;Let’s try a provisioning.&lt;/p&gt;

&lt;pre&gt;
  $ bin/vagrant up
  $ bin/vagrant ssh
&lt;/pre&gt;

&lt;p&gt;And:&lt;/p&gt;

&lt;pre&gt;
  vagrant@Ubuntu-11:~$ ruby -v
  ruby 2.0.0p0 (2013-02-24 revision 39474) [x86_64-linux]
  vagrant@Ubuntu-11:~$ rails -v
  Rails 4.0.0.beta1
&lt;/pre&gt;

&lt;p&gt;So, what do you think? There is more tuning needed, like setup of the nginx virtual host that connect to the Rails application. Also, the nginx conversion step might break, and users are missing. Any feedback would be interesting. Thanks for reading and sharing!&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>What is Erlang?</title>
   <link href="https://thinkingonthinking.com/what-is-erlang/"/>
   <updated>2013-02-20T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/what-is-erlang</id>
   <content type="html">&lt;p&gt;You will touch Erlang at some point, if you follow the development of distributed systems. Erlang is a remedy to the pains of a monolithic application. And great programmers such as &lt;a href=&quot;http://about.adamwiggins.com/&quot;&gt;Adam Wiggins&lt;/a&gt; or &lt;a href=&quot;http://plataformatec.com.br/crafting-rails-applications&quot;&gt;Jose Valim&lt;/a&gt; name Erlang in the context of scalable systems e.g. in &lt;a href=&quot;http://adam.heroku.com/past/2009/9/28/background_jobs_with_rabbitmq_and_minion/&quot;&gt;this article&lt;/a&gt; or &lt;a href=&quot;http://blog.plataformatec.com.br/2011/03/why-rubyists-should-try-elixir/&quot;&gt;here&lt;/a&gt;. So, I was curious enough to attend a great, small conference on Erlang:  &lt;a href=&quot;http://www.erlang-factory.com/conference/Munich2013&quot;&gt;Munich Erlang Factory Lite 2013&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;As someone who is new to Erlang, a first question must be asked: What is Erlang? Let’s look at different aspects.&lt;/p&gt;

&lt;h2 id=&quot;how-does-erlang-code-look-like&quot;&gt;How does Erlang code look like?&lt;/h2&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;At first, Erlang code does not look much different from Ruby, JavaScript or CoffeeScript: You’ll see traces of a dynamic type system and Erlang allows to dynamically run code with some kind of console, the Erlang Emulator (“ERL”).&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Looking more at the code, you’ll see that lines of code sometimes end with a “dot”. This triggers code execution. As a beginner, it’s the moment when you see error messages and interpretation of error messages is rather difficult. To me, Erlang errors are cryptic, and apparently line numbers in errors were a late innovation.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Further, the code you’ll see may contain tuples and atoms, some of the main data types in Erlang. Nota bene, the concept of String is not a data type in Erlang, but rather an Array of characters as in e.g. C. Some people say this makes string operations more difficult than in other languages. Other people say, it’s less error prone too. To a beginner, it’s hard to judge.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;erlang-modules-and-libraries&quot;&gt;Erlang modules and libraries&lt;/h2&gt;

&lt;p&gt;Once programming, you will quickly discover an important Erlang concept: Modules. Modules wrap functions and behavior of code. Additionally, modules can be distributed within libraries, and as in any other programming language, libraries make a programmer’s life fun.&lt;/p&gt;

&lt;p&gt;The magic command to import libraries into your Erlang project is &lt;a href=&quot;https://github.com/basho/rebar&quot;&gt;“rebar”&lt;/a&gt;. Rebar also allows to easily create a new Erlang project. And, rebar is from &lt;a href=&quot;http://basho.com/&quot;&gt;Basho&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;why-erlang&quot;&gt;Why Erlang?&lt;/h2&gt;

&lt;p&gt;So far, so good. Erlang programming is about functions and values, and that sounds like no big deal to some. However, the main aspects of Erlang are reflecting on errors when needed, and reliably running concurrent processes. It’s here where the Erlang fun part starts, and other languages just suck (for the skeptics, maybe you can read &lt;a href=&quot;http://stackoverflow.com/questions/6923480/java-thread-programming-book&quot;&gt;the books suggested here&lt;/a&gt;).&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;In Erlang, there seems to be the idea of “let it crash”. In a monolithic application, this usually means the death of an application. In a Unix system, this may mean to kill a process. In Erlang, you get a &lt;a href=&quot;http://www.erlang.org/doc/design_principles/sup_princ.html&quot;&gt;supervisor&lt;/a&gt; that may take care of this, when a process is idling or conflicting with another process. The supervisor may decide to retry, continue, or check later. This is new to me, but I haven’t come across this mentality in any other pogramming language so far.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Having better ways to deal with exceptions, you’ll also get better ways to deal with concurrency. It’s rather new to me, but to understand why this helps, think of a monolithic application that triggers delayed jobs. If you run you run multiple applications in your server cluster, and have eventually multiple delayed jobs, from my experience, you can get &lt;a href=&quot;http://en.wikipedia.org/wiki/Race_condition&quot;&gt;race conditions&lt;/a&gt; and other problems, that are really hard to debug. Erlang to the rescue. With Erlang, you’ll get some ways to have processes check what another process is doing. The mechanisms that you’ll hear people talk about are &lt;a href=&quot;http://learnyousomeerlang.com/errors-and-processes#monitors&quot;&gt;monitoring&lt;/a&gt; or linking processes, resulting into process &lt;a href=&quot;http://www.erlang.org/doc/getting_started/robustness.html&quot;&gt;robustness&lt;/a&gt;. The code of &lt;a href=&quot;http://www.rabbitmq.com/&quot;&gt;RabbitMQ&lt;/a&gt; must show clever ways to deal with message concurrency.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;conclusions&quot;&gt;Conclusions&lt;/h2&gt;

&lt;p&gt;So, my main conclusions:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;Erlang provides a dynamic toolchain that helps rethinking concurrency and fault-tolerance. Some more meta-discussions and examples on Erlang are John-Paul Bader introduction on &lt;a href=&quot;http://smyck.net/2012/04/22/why-erlang/&quot;&gt;why erlang here&lt;/a&gt;. &lt;a href=&quot;http://jlouisramblings.blogspot.de/&quot;&gt;JLouis&lt;/a&gt; ramblings on fault tolerant systems in Erlang. &lt;a href=&quot;https://github.com/basho&quot;&gt;Basho&lt;/a&gt; implemented &lt;a href=&quot;http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html&quot;&gt;Amazon’s Dynamo&lt;/a&gt; with Erlang, and the company is a great contributor and promotor to the Erlang community. And if you are not so much into web, Peer Stritzinger together with &lt;a href=&quot;http://www.embedded-brains.de/index.php?id=1&amp;amp;L=1&quot;&gt;Embedded-brains&lt;/a&gt; use Erlang for &lt;a href=&quot;http://www.stritzinger.com/Peer-Stritzinger.GmbH/Hydraprog-3.html&quot;&gt;scalable hardware&lt;/a&gt;.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Good places to get started with Erlang are John Paul’s &lt;a href=&quot;http://smyck.net/2013/02/17/how-to-set-up-a-basic-http-api-with-erlang-in-10-minutes/&quot;&gt;Setup an API in 10 minutes&lt;/a&gt; and &lt;a href=&quot;http://learnyousomeerlang.com/&quot;&gt;http://learnyousomeerlang.com/&lt;/a&gt;. Also, this blog post is a great overview: &lt;a href=&quot;http://www.nodal.info/2012/10/top-10-resources-for-getting-started.html&quot;&gt;http://www.nodal.info/2012/10/top-10-resources-for-getting-started.html&lt;/a&gt;. And &lt;a href=&quot;http://de.slideshare.net/pavlobaron/let-it-crash-seacon11-pavlo-baron&quot;&gt;Pavlo Baron&lt;/a&gt; does a lot of convincing work on &lt;a href=&quot;http://de.slideshare.net/pavlobaron/erlang-is-a-one-way&quot;&gt;using Erlang&lt;/a&gt;.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;OTP are the Erlang libraries that help in designing distributed applications. See &lt;a href=&quot;http://learnyousomeerlang.com/what-is-otp&quot;&gt;http://learnyousomeerlang.com/what-is-otp&lt;/a&gt;. The guy to ask on what’s OTP is &lt;a href=&quot;https://www.erlang-factory.com/upload/presentations/719/francesco-otp.pdf&quot;&gt;Francesco Cesarini&lt;/a&gt;.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;If you are into web development, see &lt;a href=&quot;https://github.com/elixir-lang/elixir&quot;&gt;Elixir&lt;/a&gt; and &lt;a href=&quot;https://github.com/elixir-lang/dynamo&quot;&gt;Dynamo&lt;/a&gt;. This looks the place to start writing your next hit-in-one API.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;
</content>
 </entry>
 
 <entry>
   <title>Organizing a Backbone Application</title>
   <link href="https://thinkingonthinking.com/organizing-a-backbone-application/"/>
   <updated>2013-02-15T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/organizing-a-backbone-application</id>
   <content type="html">&lt;p&gt;Organizing a frontend JavaScript stack is hard. Even more so, if you want to package and transport complete apps to the browser of a user.&lt;/p&gt;

&lt;p&gt;For example with BackboneJS, you can have multiple views referencing the same models and collections. Have a look &lt;a href=&quot;http://backbone-patterns.heroku.com/&quot;&gt;at these Backbone patterns&lt;/a&gt; by &lt;a href=&quot;http://twitter.com/julio_ody&quot;&gt;julio_ody&lt;/a&gt; to get some basics of Backbone code. Or, at &lt;a href=&quot;http://thinkingonthinking.com/fast-start-with-backbonejs/&quot;&gt;a faster start with Backbone.js&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;http://thinkingonthinking.com/what-is-modular-architecture/&quot;&gt;Making a web app modular&lt;/a&gt; is often good advice. In summary,you can avoid a spaghetti of “script” tags in HTML with JavaScript modules. JavaScript modules provide code re-use that is similar to “imports” or “requires” in other programming languages.&lt;/p&gt;

&lt;p&gt;Unfortunately, the current JavaScript implementation (ECMAScript5) does &lt;em&gt;not&lt;/em&gt; have a module loader by default. But you can add this concept with external libraries, such as &lt;a href=&quot;http://requirejs.org/&quot;&gt;RequireJS&lt;/a&gt;. Or, you can use the CommonJS module format and &lt;a href=&quot;http://thinkingonthinking.com/unix-in-the-browser/&quot;&gt;browserify modules&lt;/a&gt;. This blog post is about RequireJS and originated from a discussion with &lt;a href=&quot;https://twitter.com/netzzwerg&quot;&gt;Bela Varga&lt;/a&gt;.&lt;/p&gt;

&lt;h1 id=&quot;top-level-organization-of-a-backbone-project&quot;&gt;Top-level organization of a Backbone project&lt;/h1&gt;

&lt;p&gt;Most new programming projects start with the setup of files and directories. As Backbone projects come with no default directory structure, you can decide how to setup files and directories.&lt;/p&gt;

&lt;p&gt;In many Backbone projects, you will find a directory layout similar to:&lt;/p&gt;

&lt;pre&gt;
   |-collections
   |-libs
   |---backbone
   |---isotope
   |---jquery
   |---jquery-fileupload
   |---jquery-ui
   |---require
   |---underscore
   |-models
   |-templates
   |-views
&lt;/pre&gt;

&lt;p&gt;Separate directories for Models, Collections, Views and Templates, as well as a directory for the dependencies, work very well.&lt;/p&gt;

&lt;p&gt;If you want to learn more on how a RequireJS based setup works, &lt;a href=&quot;http://backbonetutorials.com/&quot;&gt;here&lt;/a&gt; is a nice tutorial.&lt;/p&gt;

&lt;h2 id=&quot;appjs-contains-views---not-collections&quot;&gt;App.js contains views - not collections&lt;/h2&gt;

&lt;p&gt;In a single-page application, there will be many views that need to be arranged. Now, in my first experiments, I instantiated views in the Router with:&lt;/p&gt;

&lt;pre&gt;
new SomeView(...)
&lt;/pre&gt;

&lt;p&gt;together with:&lt;/p&gt;

&lt;pre&gt;
collection.fetch(...) 
&lt;/pre&gt;

&lt;p&gt;to populate a view with data. This quickly leads to exploding dependencies in the Router. A better way to organize views is the following:&lt;/p&gt;

&lt;pre&gt;
    var AppRouter = Backbone.Router.extend({
        routes: {
	      &apos;&apos;: &apos;showPostListView&apos;,
	      &apos;/posts/:id&apos;: &apos;showPostDetailView&apos;,
        },

        showPostListView: function() {
	      new PostListView();
        },

       	showPostDetailView: function(post) {
	      new PostDetailView({id: post});
        }
    });
&amp;lt;/div&amp;gt;
&lt;/pre&gt;

&lt;p&gt;The idea is that the Views know which collection and models they bind too, and whether to fetch data or not. Here, we don’t need to require any Collection or Model since this will be done from within a view.&lt;/p&gt;

&lt;h2 id=&quot;naming-views&quot;&gt;Naming Views&lt;/h2&gt;
&lt;p&gt;Naming is always hard. And I found naming things on the frontend to be slightly different from naming things in the backend. In the Backend, you might want to orientate at naming things to entities from a domain; in a single-page app, you might want to name things after how you want to show or interact with data.&lt;/p&gt;

&lt;p&gt;For example, one common frontend pattern is to show and interact with items in a collection. It’s what the basic Backbone &lt;a href=&quot;http://backbonejs.org/examples/todos/index.html&quot;&gt;ToDo&lt;/a&gt; shows, but what I understand now is the importance of naming:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;A ‘ListView’ identifies a view that belongs to a collection.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;An ‘ItemView’ identifies a view that belogns to a model.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Now, when you break up your single-page app into different views, having this simple rule above helps in structuring the different states of the frontend. Also, ListView are slightly less bound to templates, but rather act as parent views for different item sub-views.&lt;/p&gt;

&lt;h2 id=&quot;conclusions&quot;&gt;Conclusions&lt;/h2&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;For the main structure of a Backbone application, I find the directories /collections, /models, /views/ and /templates helpful.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Start thinking in Views first (a good option is on paper). Then, think on how Views fetch data when needed.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;There are views that manage other Views, and Views that render templates.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;There seems to be some conventions on naming views in Backbone. So far, I use ‘ListView’ and ‘ItemView’ to name views on collections, or the elements (= items) in a collection.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;
</content>
 </entry>
 
 <entry>
   <title>Transforming collections in Ruby</title>
   <link href="https://thinkingonthinking.com/map-reduce-in-ruby/"/>
   <updated>2013-02-09T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/map-reduce-in-ruby</id>
   <content type="html">&lt;p&gt;Recently, I’ve read some strategy to visualize user registrations and activity in a web application with Ruby and Google Charts in &lt;a href=&quot;http://gistflow.com/posts/670-visualising-database-1&quot;&gt;this article&lt;/a&gt; . After starting my own visualization experiments, I felt that I had some blind spots in understanding Ruby’s &lt;a href=&quot;http://ruby-doc.org/core-1.9.3/Enumerable.html&quot;&gt;Enumerable Module&lt;/a&gt;. This post attempts to share some insights on expanding and reducing Arrays of objects.&lt;/p&gt;

&lt;p&gt;First, let’s assume our data looks like:&lt;/p&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
signups = [&quot;2011-08&quot;=&amp;gt;1433, &quot;2011-09&quot;=&amp;gt;2972, &quot;2011-10&quot;=&amp;gt;2480, &quot;2011-11&quot;=&amp;gt;2456]
logins = [&quot;2011-08&quot;=&amp;gt;2832, &quot;2011-09&quot;=&amp;gt;4172, &quot;2011-10&quot;=&amp;gt;3480, &quot;2011-11&quot;=&amp;gt;5456]
&lt;/pre&gt;

&lt;p&gt;My goal was to combine the two collections and transform the result into:&lt;/p&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
activity = [ &quot;2011-08&quot; =&amp;gt; { :signups =&amp;gt; 1433, :logins =&amp;gt; 2832 } ...
&lt;/pre&gt;

&lt;p&gt;In the exploration, I touched questions on #merge and #to_a of Hashes combined with #first and #last of Arrays, and #map, #inject, #group_by for Enumerables. Let’s start.&lt;/p&gt;

&lt;h2 id=&quot;splitting-a-hash&quot;&gt;Splitting a Hash&lt;/h2&gt;
&lt;p&gt;My first step was to split the larger collection hash into single hashes. For this &lt;a href=&quot;http://ruby-doc.org/core-1.9.3/Enumerable.html#method-i-first&quot;&gt;first&lt;/a&gt; and &lt;a href=&quot;http://ruby-doc.org/core-1.9.3/Enumerable.html#method-i-to_a&quot;&gt;to_a&lt;/a&gt; methods help like this:&lt;/p&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
signups.first.to_a
=&amp;gt; [[&quot;2011-08&quot;, 1433], [&quot;2011-09&quot;, 2972], [&quot;2011-10&quot;, 2480], [&quot;2011-11&quot;, 2456]]
&lt;/pre&gt;

&lt;p&gt;There is one caveat with inverse transformation, i.e. converting above Array back to Hashes. I knew that:&lt;/p&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
Hash[1,2]
=&amp;gt; {1 =&amp;gt; 2}
&lt;/pre&gt;

&lt;p&gt;but this does not work:&lt;/p&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
[[&quot;2011-08&quot;, 1433], [&quot;2011-09&quot;, 2972], [&quot;2011-10&quot;, 2480], [&quot;2011-11&quot;, 2456]].map { |a| Hash[a] }
=&amp;gt; [{}, {}, {}, {}]
&lt;/pre&gt;

&lt;p&gt;In order to have the conversion working, the arguments must be ‘splatted’ with *, see e.g. &lt;a href=&quot;http://theplana.wordpress.com/2007/03/03/ruby-idioms-the-splat-operator/&quot;&gt;this discussion&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;So, this means by splating the input values with Hash[*a] we get:&lt;/p&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
signups = [[&quot;2011-08&quot;, 1433], [&quot;2011-09&quot;, 2972], [&quot;2011-10&quot;, 2480], [&quot;2011-11&quot;, 2456]].map { |a| Hash[*a] }
=&amp;gt; [{&quot;2011-08&quot;=&amp;gt;1433}, {&quot;2011-09&quot;=&amp;gt;2972}, {&quot;2011-10&quot;=&amp;gt;2480}, {&quot;2011-11&quot;=&amp;gt;2456}]
&lt;/pre&gt;

&lt;p&gt;Now we have an Array of Hashes for the signups (while we started with one Hash wrapped in an Array)
As I want some key in the collection that I could use later for grouping, let’s introduce a :signup symbol as key like this:&lt;/p&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
signups.map { |h| h.map { |k, v| { k =&amp;gt; { signups: v }}}}.flatten
=&amp;gt; [{&quot;2011-08&quot;=&amp;gt;{:signups=&amp;gt;1433}}, {&quot;2011-09&quot;=&amp;gt;{:signups=&amp;gt;2972}}, {&quot;2011-10&quot;=&amp;gt;{:signups=&amp;gt;2480}}, {&quot;2011-11&quot;=&amp;gt;{:signups=&amp;gt;2456}}]
&lt;/pre&gt;

&lt;p&gt;Notice, there are two &lt;a href=&quot;http://ruby-doc.org/core-1.9.3/Enumerable.html#method-i-map&quot;&gt;#map&lt;/a&gt; calls here: The first #map is about walking on the elements of an Array, while the second #map walks on the key and values of a Hash. For me, a common mistoke is trying to walk an Array with pair elements, while these are only available in the Hash #map.&lt;/p&gt;

&lt;p&gt;Let’s apply a the transformation so far also to our second collection, the logins collections with:&lt;/p&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
[&quot;2011-08&quot;=&amp;gt;2832, &quot;2011-09&quot;=&amp;gt;4172, &quot;2011-10&quot;=&amp;gt;3480, &quot;2011-11&quot;=&amp;gt;5456].first.to_a.map { |a| Hash[*a] }.map { |h| h.map { |k, v| { k =&amp;gt; { logins: v }}}}.flatten
=&amp;gt; [{&quot;2011-08&quot;=&amp;gt;{:logins=&amp;gt;2832}}, {&quot;2011-09&quot;=&amp;gt;{:logins=&amp;gt;4172}}, {&quot;2011-10&quot;=&amp;gt;{:logins=&amp;gt;3480}}, {&quot;2011-11&quot;=&amp;gt;{:logins=&amp;gt;5456}}]
&lt;/pre&gt;

&lt;h2 id=&quot;reduce-and-grouping&quot;&gt;Reduce and Grouping&lt;/h2&gt;
&lt;p&gt;The steps above were basically joining and expanding our collection that should now be reduced back to the final structure. To get this new, combined collection, let’s summarize the structure we have so far:&lt;/p&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
signups = [{&quot;2011-08&quot;=&amp;gt;{:signups=&amp;gt;1433}}, {&quot;2011-09&quot;=&amp;gt;{:signups=&amp;gt;2972}}, {&quot;2011-10&quot;=&amp;gt;{:signups=&amp;gt;2480}}, {&quot;2011-11&quot;=&amp;gt;{:signups=&amp;gt;2456}}]
logins = [{&quot;2011-08&quot;=&amp;gt;{:logins=&amp;gt;2832}}, {&quot;2011-09&quot;=&amp;gt;{:logins=&amp;gt;4172}}, {&quot;2011-10&quot;=&amp;gt;{:logins=&amp;gt;3480}}, {&quot;2011-11&quot;=&amp;gt;{:logins=&amp;gt;5456}}]
&lt;/pre&gt;

&lt;p&gt;Before we are going to mix up the collections, let’s shortly review how #reduce works.&lt;/p&gt;

&lt;h3 id=&quot;reduce-or-inject&quot;&gt;Reduce (or Inject)&lt;/h3&gt;

&lt;p&gt;The methods #reduce and #inject do the same (they are aliased). The process is sometimes known as ‘folding’ a collection (see &lt;a href=&quot;http://en.wikipedia.org/wiki/Fold_(higher-order_function)&quot;&gt;wikipedia&lt;/a&gt;). Here are two observations I made:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Reduce let apply operators in an Enumerable list:&lt;/li&gt;
&lt;/ul&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
[1,2,3].reduce(:+)
=&amp;gt; 6
&lt;/pre&gt;

&lt;ul&gt;
  &lt;li&gt;The effect of Reduce is a bit like chaining method calls:&lt;/li&gt;
&lt;/ul&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
[{a: 1},{b:2}].reduce(:merge)
=&amp;gt; {:a=&amp;gt;1, :b=&amp;gt;2}
&lt;/pre&gt;

&lt;p&gt;This reminds me on {:a=&amp;gt;1}.merge({:b=&amp;gt;2}) that has the same outcome: {:a=&amp;gt;1, :b=&amp;gt;2}&lt;/p&gt;

&lt;p&gt;Inject is very handy to walk a list, and at the same time keep some notes in memory on the progress. Sometimes an additional step might be necessary, to apply folding, e.g. prepare the collection with &lt;a href=&quot;http://ruby-doc.org/core-1.9.3/Enumerable.html#method-i-group_by&quot;&gt;#group_by&lt;/a&gt;.&lt;/p&gt;

&lt;h3 id=&quot;group_by&quot;&gt;Group_by&lt;/h3&gt;

&lt;p&gt;My observation on the #group_by method is that it allows to group data according to ‘meta-data’. Consider grouping a number of objects according to their class. For example:&lt;/p&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
[&quot;12&quot;, 12, &quot;patrick&quot;, Object.new].group_by(&amp;amp;:class)
=&amp;gt; {String=&amp;gt;[&quot;12&quot;, &quot;patrick&quot;], Fixnum=&amp;gt;[12], Object =&amp;gt; ...]}
&lt;/pre&gt;

&lt;p&gt;Now, we could also group this collection according to their String values like this:&lt;/p&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
[&quot;12&quot;, 12, &quot;patrick&quot;, Object.new].group_by(&amp;amp;:to_s)
=&amp;gt; {&quot;12&quot;=&amp;gt;[&quot;12&quot;, 12], &quot;patrick&quot;=&amp;gt;[&quot;patrick&quot;], Object:0x007f =&amp;gt;[...&amp;gt;]}
&lt;/pre&gt;

&lt;p&gt;The same data, different groupings. However, so far we can go ahead again with our signup and login collections, and apply some #inject calls.&lt;/p&gt;

&lt;h3 id=&quot;applying-inject&quot;&gt;Applying #inject&lt;/h3&gt;

&lt;p&gt;My first goal is to transform a structure as this:&lt;/p&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
[{ :a =&amp;gt; { :b =&amp;gt; 10 }}, { :a =&amp;gt; { :c =&amp;gt; 20 }}]
&lt;/pre&gt;

&lt;p&gt;into:&lt;/p&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
[{ :a =&amp;gt; { :b =&amp;gt; 10, :c =&amp;gt; 20 }}]
&lt;/pre&gt;

&lt;p&gt;This can be done with:&lt;/p&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
[{:a=&amp;gt;{:b=&amp;gt;10}}, {:a=&amp;gt;{:c=&amp;gt;20}}].map { |k| k.values }.flatten.reduce(:merge)
=&amp;gt; {:b=&amp;gt;10, :c=&amp;gt;20}
&lt;/pre&gt;

&lt;p&gt;The explanation: We reduce an Array of a nested Hash with the values from the start.&lt;/p&gt;

&lt;p&gt;Our next step is to remove the extra Hashes here:&lt;/p&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
[{ :a =&amp;gt; { :b =&amp;gt; 10, :c =&amp;gt; 20 }}, { :d =&amp;gt; { :b =&amp;gt; 10, :c =&amp;gt; 20 }}]
&lt;/pre&gt;

&lt;p&gt;This can be done with:&lt;/p&gt;
&lt;pre style=&quot;font-size:11px&quot;&gt;
[{ :a =&amp;gt; { :b =&amp;gt; 10, :c =&amp;gt; 20 }}, { :d =&amp;gt; { :b =&amp;gt; 10, :c =&amp;gt; 20 }}].reduce(:merge)
=&amp;gt; {:a=&amp;gt;{:b=&amp;gt;10, :c=&amp;gt;20}, :d=&amp;gt;{:b=&amp;gt;10, :c=&amp;gt;20}}
&lt;/pre&gt;

&lt;p&gt;Be careful: With a minor change the values from the first Hash are lost:&lt;/p&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
[{ :a =&amp;gt; { :b =&amp;gt; 10, :c =&amp;gt; 20 }}, { :a =&amp;gt; { :f =&amp;gt; 10, :d =&amp;gt; 20 }}].reduce(:merge)
=&amp;gt; {:a=&amp;gt;{:f=&amp;gt;10, :d=&amp;gt;20}}
&lt;/pre&gt;

&lt;p&gt;Note: The reason that the merge does not work as expected is, that a merge does not walk recursively into a nested Hash. To get the proper merge operation, we need to apply #inject with an empty hash as accumulator like this:&lt;/p&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
[{ :a =&amp;gt; { :b =&amp;gt; 10, :c =&amp;gt; 20 }}, { :a =&amp;gt; { :f =&amp;gt; 10, :d =&amp;gt; 20 }}].inject({}) {|o,h| o.merge!(h[:a]); o }
=&amp;gt; {:b=&amp;gt;10, :c=&amp;gt;20, :f=&amp;gt;10, :d=&amp;gt;20}
&lt;/pre&gt;

&lt;p&gt;Now, a similar transformation can be applied to my joined signup and login collection. First, let’s combine the collections:&lt;/p&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
[logins, signups].flatten.group_by { |a| a.keys.first }
=&amp;gt; {&quot;2011-08&quot;=&amp;gt;[{&quot;2011-08&quot;=&amp;gt;{:logins=&amp;gt;2832}}, {&quot;2011-08&quot;=&amp;gt;{:signups=&amp;gt;1433}}], &quot;2011-09&quot;=&amp;gt;[{&quot;2011-09&quot;=&amp;gt;{:logins=&amp;gt;4172}}, {&quot;2011-09&quot;=&amp;gt;{:signups=&amp;gt;2972}}], &quot;2011-10&quot;=&amp;gt;[{&quot;2011-10&quot;=&amp;gt;{:logins=&amp;gt;3480}}, {&quot;2011-10&quot;=&amp;gt;{:signups=&amp;gt;2480}}], &quot;2011-11&quot;=&amp;gt;[{&quot;2011-11&quot;=&amp;gt;{:logins=&amp;gt;5456}}, {&quot;2011-11&quot;=&amp;gt;{:signups=&amp;gt;2456}}]}
&lt;/pre&gt;

&lt;p&gt;Now, let’s reduce the set to the basics:&lt;/p&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
login_signups = [logins, signups].flatten.inject({}) { |h,v| h[v.keys.first] ||= []; h[v.keys.first] &amp;lt;&amp;lt; v.values; puts v.inspect; h}
{&quot;2011-08&quot;=&amp;gt;{:logins=&amp;gt;2832}}
{&quot;2011-09&quot;=&amp;gt;{:logins=&amp;gt;4172}}
{&quot;2011-10&quot;=&amp;gt;{:logins=&amp;gt;3480}}
{&quot;2011-11&quot;=&amp;gt;{:logins=&amp;gt;5456}}
{&quot;2011-08&quot;=&amp;gt;{:signups=&amp;gt;1433}}
{&quot;2011-09&quot;=&amp;gt;{:signups=&amp;gt;2972}}
{&quot;2011-10&quot;=&amp;gt;{:signups=&amp;gt;2480}}
{&quot;2011-11&quot;=&amp;gt;{:signups=&amp;gt;2456}}
=&amp;gt; {&quot;2011-08&quot;=&amp;gt;[[{:logins=&amp;gt;2832}], [{:signups=&amp;gt;1433}]], &quot;2011-09&quot;=&amp;gt;[[{:logins=&amp;gt;4172}], [{:signups=&amp;gt;2972}]], &quot;2011-10&quot;=&amp;gt;[[{:logins=&amp;gt;3480}], [{:signups=&amp;gt;2480}]], &quot;2011-11&quot;=&amp;gt;[[{:logins=&amp;gt;5456}], [{:signups=&amp;gt;2456}]]}
&lt;/pre&gt;

&lt;p&gt;Now, we are almost there. Let’s clean up the combined hashes and arrays within the date keys:&lt;/p&gt;

&lt;pre style=&quot;font-size:11px&quot;&gt;
activities = login_signups.map { |m| {m.first =&amp;gt; m.last.flatten.inject(:merge)} }=&amp;gt; [{&quot;2011-08&quot;=&amp;gt;{:logins=&amp;gt;2832, :signups=&amp;gt;1433}}, {&quot;2011-09&quot;=&amp;gt;{:logins=&amp;gt;4172, :signups=&amp;gt;2972}}, {&quot;2011-10&quot;=&amp;gt;{:logins=&amp;gt;3480, :signups=&amp;gt;2480}}, {&quot;2011-11&quot;=&amp;gt;{:logins=&amp;gt;5456, :signups=&amp;gt;2456}}]
=&amp;gt; [{&quot;2011-08&quot;=&amp;gt;{:logins=&amp;gt;2832, :signups=&amp;gt;1433}}, {&quot;2011-09&quot;=&amp;gt;{:logins=&amp;gt;4172, :signups=&amp;gt;2972}}, {&quot;2011-10&quot;=&amp;gt;{:logins=&amp;gt;3480, :signups=&amp;gt;2480}}, {&quot;2011-11&quot;=&amp;gt;{:logins=&amp;gt;5456, :signups=&amp;gt;2456}}]
&lt;/pre&gt;

&lt;h3 id=&quot;conclusion&quot;&gt;Conclusion&lt;/h3&gt;

&lt;p&gt;My conclusion of this post are the following:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;Transforming nested collections is funcitonal programming, and it certainly takes practice. I would be curious to see/hear how you approach this kind of problems with your steps, or maybe in another programming language. Let me know what you think.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;A lot of transformation can be approached with standard Array methods like ‘each’ and ‘map’; however they are sometimes not optimal, and in a way are limited to ‘simple’ transformation only.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Redis as key-value store provides commands for Hash, List and Set operations; maybe getting a better understanding of the effects in Ruby helps you becoming a better data engineer too.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;I didn’t use #group_by; although it is very helpful in a collections with more keys, e.g. see this discussion on &lt;a href=&quot;http://stackoverflow.com/questions/14776686/ruby-transformation-to-merge-an-array-of-hashes-into-another-array-of-hash&quot;&gt;StackOverflow&lt;/a&gt;&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Probably a nice topic for exploration would also be using value objects for this kind of filtering and collection transformation&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;
</content>
 </entry>
 
 <entry>
   <title>Logging in Rack middleware with Clogger</title>
   <link href="https://thinkingonthinking.com/log-requests-with-rack-middleware-and-clogger/"/>
   <updated>2013-02-05T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/log-requests-with-rack-middleware-and-clogger</id>
   <content type="html">&lt;p&gt;Thanks to Rack middleware and &lt;a href=&quot;http://api.rubyonrails.org/files/railties/lib/rails/engine_rb.html&quot;&gt;Rails engines&lt;/a&gt;, it is possible to easily combine web application modules in Ruby. Good examples of this pattern are the &lt;a href=&quot;https://github.com/spree/spree&quot;&gt;Spree engine&lt;/a&gt; or some chapters of Josè Valim’s book &lt;a href=&quot;http://plataformatec.com.br/crafting-rails-applications&quot;&gt;“Crafting Rails Applications”&lt;/a&gt;. In the context of the design of &lt;a href=&quot;http://thinkingonthinking.com/what-is-modular-architecture/&quot;&gt;modular web applications&lt;/a&gt;, I found logging specifics of HTTP requests with &lt;a href=&quot;http://clogger.rubyforge.org/&quot;&gt;Clogger&lt;/a&gt; very helpful.&lt;/p&gt;

&lt;h2 id=&quot;what-is-clogger&quot;&gt;What is Clogger?&lt;/h2&gt;
&lt;p&gt;The main function of Clogger is to print and format parameters in a &lt;a href=&quot;http://rack.rubyforge.org/doc/Rack/Request.html&quot;&gt;Rack::Request&lt;/a&gt;. It’s a bit like ‘sprintf’ for HTTP requests, and can be mounted as middleware in a Rack application. So far, I mainly used the logger with Sinatra or in Rails application. Here are 2 examples:&lt;/p&gt;

&lt;h2 id=&quot;logging-http-headers-in-a-rails-application&quot;&gt;Logging HTTP headers in a Rails application&lt;/h2&gt;
&lt;p&gt;Authentication over the web can be tricky. For development purposes, it’s often helpful to see some more details of a HTTP header. For this, let’s write a simple client sending some header tokens to a Rails app.&lt;/p&gt;

&lt;p&gt;The client:&lt;/p&gt;

&lt;pre&gt;
require &apos;faraday&apos;
client = Faraday::Connection.new(:url =&amp;gt; &quot;http://0.0.0.0:3000/&quot;)
client.get &apos;/example&apos; do |req|
  req.headers[&apos;x_token&apos;] = &apos;some meta data&apos;
end
&lt;/pre&gt;

&lt;p&gt;The Clogger setup in a Rails server config/application.rb:&lt;/p&gt;

&lt;pre style=&quot;font-size:10px&quot;&gt;
class Application &amp;lt; Rails::Application
  config.middleware.use Clogger, :format =&amp;gt; &apos;&quot;$request&quot; ? $query_string $status $body_bytes_sent &quot;$http_referer&quot; &quot;$http_user_agent&quot; $request_time -- $http_x_token&apos;, :logger =&amp;gt; $stdout 
end
&lt;/pre&gt;

&lt;p&gt;Results in:&lt;/p&gt;

&lt;pre&gt;
  &quot;GET http://0.0.0.0:3000/profiles HTTP/1.1&quot; ?  200 1083 &quot;-&quot; &quot;Ruby&quot; 0.502 -- some meta data
&lt;/pre&gt;

&lt;h2 id=&quot;monitoring-cookies-in-sinatra&quot;&gt;Monitoring cookies in Sinatra&lt;/h2&gt;
&lt;p&gt;A similar logging setup can easily be done in Sinatra. Simply include Clogger in config.ru&lt;/p&gt;

&lt;pre&gt;
require &apos;clogger&apos;

use Clogger, :format =&amp;gt; &apos;&quot;$request&quot; ?$query_string $status &quot;$http_cookie&quot; $request_time $http_host&apos;, :logger =&amp;gt; $stdout
&lt;/pre&gt;

&lt;p&gt;Now the cookies are dumped for every request. This can be helpful for debugging once in a while.&lt;/p&gt;

&lt;h2 id=&quot;conclusion&quot;&gt;Conclusion&lt;/h2&gt;
&lt;p&gt;The take away from this post is that the Rack middleware can be a nice tool for debugging HTTP header content. Rack can bring modular code back to a web application, and this is great to create ‘layers’ of functionality upon which other applications can build on. A similar approach was derived in the Faraday stack to write &lt;a href=&quot;http://adventuresincoding.com/2010/09/writing-modular-http-client-code-with-faraday/&quot;&gt;modular client-side middleware&lt;/a&gt;, and will may be discussed in a later post.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>A Vagrant based Minimum Viable Webserver</title>
   <link href="https://thinkingonthinking.com/minimum-nginx-node-with-librarian-chef/"/>
   <updated>2013-02-01T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/minimum-nginx-node-with-librarian-chef</id>
   <content type="html">&lt;p&gt;When developing servers or clients that talk HTTP, it’s great to have mock systems that are as close as possible to a live system. Here is a quick hack to make a minimum viable webserver by using Chef-Solo and &lt;a href=&quot;https://github.com/applicationsonline/librarian&quot;&gt;Librarian-Chef&lt;/a&gt;. Librarian-Chef is conceptually very close to the Ruby Bundler gem.&lt;/p&gt;

&lt;h2 id=&quot;install-librarian-chef&quot;&gt;Install librarian-chef&lt;/h2&gt;

&lt;pre&gt;
   $ gem install librarian
&lt;/pre&gt;

&lt;h2 id=&quot;init-librarian-chef&quot;&gt;Init librarian-chef&lt;/h2&gt;

&lt;p&gt;To get started, we need a file where we can specifiy a node’s cookbooks:&lt;/p&gt;

&lt;pre&gt;
   $ librarian-chef init
&lt;/pre&gt;

&lt;p&gt;This gives us the ‘Cheffile’, which is conceptually close to Bundler’s ‘Gemfile’.&lt;/p&gt;

&lt;h2 id=&quot;edit-cheffile&quot;&gt;Edit Cheffile&lt;/h2&gt;

&lt;p&gt;We are going to modify the Cheffile to define our node’s dependencies as follows:&lt;/p&gt;

&lt;pre&gt;
#!/usr/bin/env ruby
#^syntax detection

site &apos;http://community.opscode.com/api/v1&apos;

cookbook &apos;nginx-app&apos;, :path =&amp;gt; &quot;./application-cookbooks&quot;
&lt;/pre&gt;

&lt;p&gt;This means we are going to write a small application cookbook ‘nginx-app’ that lives in the path ‘/application-cookbooks’&lt;/p&gt;

&lt;h2 id=&quot;create-the-application-cookbook&quot;&gt;Create the application cookbook&lt;/h2&gt;

&lt;p&gt;In contrast to cookbooks for infrastructure, application cookbooks are adjustments and setups for node specific stuff. Some background on this can be found &lt;a href=&quot;http://devopsanywhere.blogspot.de/2012/11/how-to-write-reusable-chef-cookbooks.html&quot;&gt;here&lt;/a&gt; They are derived from e.g. community cookbooks, but contain custom stuff. So, let’s create an application cookbook for our webserver:&lt;/p&gt;

&lt;pre&gt;
    knife cookbook create nginx-app -o application-cookbooks
&lt;/pre&gt;

&lt;p&gt;(Eventually, you need to have knife-solo on your system to get this command to work)&lt;/p&gt;

&lt;p&gt;This should give something like:&lt;/p&gt;

&lt;pre&gt;
  application-cookbooks
   |-nginx-app
   |---attributes
   |---definitions
   |---files
   |-----default
   |---libraries
   |---providers
   |---recipes
   |---resources
   |---templates
   |-----default
&lt;/pre&gt;

&lt;h2 id=&quot;setup-the-webservers-resources&quot;&gt;Setup the webserver’s resources&lt;/h2&gt;

&lt;p&gt;Nginx can be installed with a simple &lt;a href=&quot;http://wiki.opscode.com/display/chef/Resources#Resources-Package&quot;&gt;package resource&lt;/a&gt;. This is an abstraction around a package installer such as aptitude or yum. Also we need to use a template resource for defining a virtual host:&lt;/p&gt;

&lt;h3 id=&quot;recipe-in-nginx-apprecipesdefaultrb&quot;&gt;Recipe in /nginx-app/recipes/default.rb&lt;/h3&gt;

&lt;pre&gt;
   package &quot;nginx&quot;
   
   service &quot;nginx&quot; do
     supports :status =&amp;gt; true, :restart =&amp;gt; true, :reload =&amp;gt; true
     action [:enable, :start]
   end
   
   template &quot;/etc/nginx/nginx.conf&quot; do
     notifies :reload, &quot;service[nginx]&quot;
   end
&lt;/pre&gt;

&lt;h3 id=&quot;template-in-nginx-apptemplatesnginxconferb&quot;&gt;Template in /nginx-app/templates/nginx.conf.erb&lt;/h3&gt;
&lt;pre&gt;
  user www-data;
  worker_processes  &amp;lt;%= @node[:nginx][:worker_processes] %&amp;gt;;
  
  error_log  /var/log/nginx/error.log;
  pid        /var/run/nginx.pid;
  
  events {
      worker_connections  1024;
      # multi_accept on;
  }
  
  http {
      include       /etc/nginx/mime.types;
  
      access_log	/var/log/nginx/access.log;
  
      sendfile        on;
      #tcp_nopush     on;
  
      keepalive_timeout  5;
      tcp_nodelay        on;
  
      gzip  on;
      gzip_disable &quot;MSIE [1-6]\.(?!.*SV1)&quot;;
  
      include /etc/nginx/conf.d/*.conf;
      include /etc/nginx/sites-enabled/*;
  }

&lt;/pre&gt;

&lt;p&gt;We use an attribute above for the worker, so we also need to define that variable:&lt;/p&gt;

&lt;pre&gt;
  default[:nginx][:dir] = &quot;/etc/nginx&quot;
  default[:nginx][:worker_processes] = 4
&lt;/pre&gt;

&lt;h2 id=&quot;bundle-the-cookbooks&quot;&gt;Bundle the cookbooks&lt;/h2&gt;

&lt;p&gt;Now, we have everything in place and we can bundle the cookbooks with the command:&lt;/p&gt;

&lt;pre&gt;
    $ librarian-chef install
&lt;/pre&gt;

&lt;p&gt;Now all cookbooks are downloaded an placed into the local /cookbooks path. Two outputs can be observed, the dependencies were written into Cheffile.lock&lt;/p&gt;

&lt;pre&gt;
SITE
  remote: http://community.opscode.com/api/v1
  specs:
    build-essential (1.3.2)
    nginx (1.2.0)
      build-essential (&amp;gt;= 0.0.0)
      ohai (&amp;gt;= 1.1.4)
    ohai (1.1.6)

PATH
  remote: ./application-cookbooks
  specs:
    nginx-app (0.1.0)
      nginx (&amp;gt;= 0.0.0)

DEPENDENCIES
  nginx-app (&amp;gt;= 0
&lt;/pre&gt;

&lt;p&gt;This shows that our node get some additional cookbooks that are required by the nginx-app cookbook.
Let’s also look at the cookbook path:&lt;/p&gt;

&lt;pre&gt;
   cookbooks
   |-build-essential
   |---attributes
   |---recipes
   |-nginx
   |---attributes
   |---definitions
   |---files
   |-----default
   |-------tests
   |---------minitest
   |-----------support
   |---recipes
   |---templates
   |-----debian
   |-----default
   |-------modules
   |-------plugins
   |-----gentoo
   |-----ubuntu
   |-nginx-app
   |---attributes
   |---definitions
   |---files
   |-----default
   |---libraries
   |---providers
   |---recipes
   |---resources
   |---templates
   |-----default
   |-ohai
   |---attributes
   |---files
   |-----default
   |-------plugins
   |---recipes
&lt;/pre&gt;

&lt;p&gt;As we can see, all cookbooks were bundled.&lt;/p&gt;

&lt;h2 id=&quot;setup-vagrant&quot;&gt;Setup Vagrant&lt;/h2&gt;
&lt;p&gt;Now, we have all things in place to provision the web server. It’s time to setup Vagrant:&lt;/p&gt;

&lt;pre&gt;
    $ vagrant init Ubuntu-11.04
&lt;/pre&gt;

&lt;p&gt;Edit the Vagrantfile:&lt;/p&gt;
&lt;pre&gt;

&lt;pre&gt;
Vagrant::Config.run do |config|
  config.vm.box = &quot;Ubuntu-11.04&quot;
  config.vm.network :hostonly, &quot;192.168.33.10&quot;
  config.vm.forward_port 80, 8080
  config.vm.forward_port 9292, 9292

  config.vm.provision :chef_solo do |chef|
    chef.cookbooks_path = &quot;./cookbooks&quot;
    chef.add_recipe &quot;nginx-app&quot;
  end
end
&lt;/pre&gt;

And provision:
&lt;pre&gt;
  $ vagrant up
&lt;/pre&gt;

So, that&apos;s all to get the webserver running. Now, it would be cool to get some basic Ruby/Rack interface running too. Maybe in the next post with the help of a unicorn cookbook.
&lt;/pre&gt;
</content>
 </entry>
 
 <entry>
   <title>Building a Vagrant base box</title>
   <link href="https://thinkingonthinking.com/building-a-Vagrant-base-box/"/>
   <updated>2013-01-31T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/building-a-Vagrant-base-box</id>
   <content type="html">&lt;p&gt;As explained in my previous &lt;a href=&quot;http://thinkingonthinking.com/building-infrastructure&quot;&gt;post&lt;/a&gt;, Chef allows to develop, test and experiment with computing or cloud infrastructure.&lt;/p&gt;

&lt;p&gt;In order to locally build nodes to test the setup of a node, &lt;a href=&quot;http://vagrantup.com&quot;&gt;Vagrant&lt;/a&gt; is a great automation tool that helps to play with ISO images from a certain operating system. There is a list of ISO images ready for Chef here at &lt;a href=&quot;http://vagrantbox.es&quot;&gt;Vagrantbox.es&lt;/a&gt;. However, a common use case is, that you are running some other Linux version already, or, you want to work with fresh Ubuntu image, that you know what it contains.&lt;/p&gt;

&lt;p&gt;So, how to create a new Vagrant basebox?&lt;/p&gt;

&lt;p&gt;Here are some steps:&lt;/p&gt;

&lt;h2 id=&quot;1--upgrade-your-virtualbox-to-the-latest-version&quot;&gt;1.  Upgrade your VirtualBox to the latest Version&lt;/h2&gt;

&lt;p&gt;As time of writing this was 4.2.6. This gives you less headache when Vagrant is downloading so called ISO image additions (see e.g. TimeOut issue &lt;a href=&quot;https://github.com/jedi4ever/veewee/issues/519&quot;&gt;here&lt;/a&gt; )&lt;/p&gt;

&lt;h2 id=&quot;2--veewee-and-vagrant&quot;&gt;2.  VeeWee and Vagrant;&lt;/h2&gt;

&lt;p&gt;Make sure you have the gems VeeWee and Vagrant installed:&lt;/p&gt;

&lt;pre&gt;
         gem install vagrant
         gem install veewee
&lt;/pre&gt;

&lt;h2 id=&quot;3--list-baseboxes&quot;&gt;3.  List baseboxes&lt;/h2&gt;

&lt;p&gt;Now, You’ll want to get a list with possible ISO images for baseboxes. This command list baseboxes:&lt;/p&gt;

&lt;pre&gt;
        vagrant basebox templates
&lt;/pre&gt;

&lt;h2 id=&quot;4--define-a-basebox&quot;&gt;4.  Define a basebox&lt;/h2&gt;

&lt;p&gt;As you’ll see in the list you have quite a lot of options. Do a:&lt;/p&gt;

&lt;pre&gt;
     vagrant basebox define Ubuntu-11.04 ubuntu-11.04-server-amd64 -d
&lt;/pre&gt;

&lt;h2 id=&quot;5-check-the-iso-image&quot;&gt;5. Check the ISO Image&lt;/h2&gt;

&lt;p&gt;Check/Edit the definition of the base box. In my case I had to change the pointer to the ISO image in:&lt;/p&gt;

&lt;pre&gt;
    ubuntu-base/definitions/Ubuntu-11.04/definition.rb
&lt;/pre&gt;

&lt;h2 id=&quot;6--build-the-basebox&quot;&gt;6.  Build the basebox&lt;/h2&gt;

&lt;p&gt;Start the build of the basebox:&lt;/p&gt;

&lt;pre&gt;
     vagrant basebox build Ubuntu-11.04
&lt;/pre&gt;

&lt;h2 id=&quot;7--export-the-basebox&quot;&gt;7.  Export the basebox:&lt;/h2&gt;

&lt;pre&gt;
   vagrant basebox export Ubuntu-11.04
&lt;/pre&gt;

&lt;h2 id=&quot;8--add-the-basebox&quot;&gt;8.  Add the basebox:&lt;/h2&gt;

&lt;pre&gt;
   vagrant box add &apos;Ubuntu-11.04&apos; &apos;Ubuntu-11.04.box&apos;
&lt;/pre&gt;

&lt;p&gt;That’s basically it.&lt;/p&gt;

&lt;h1 id=&quot;references&quot;&gt;References&lt;/h1&gt;

&lt;ul&gt;
  &lt;li&gt;Thanks to &lt;a href=&quot;http://zac.sprackett.com/resume/&quot;&gt;Zac Sprackett&lt;/a&gt; for writing on setting up a CentOS basebox  &lt;a href=&quot;http://devops.me/2011/10/06/building-baseboxes/&quot;&gt;here&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;http://blog.eriksen.com.br/en/creating-custom-vagrant-boxes-veewee&quot;&gt;blog.eriksen.com.br/&lt;/a&gt;: Mentioning Postinstall scripts&lt;/li&gt;
  &lt;li&gt;https://github.com/opscode/bento&lt;/li&gt;
&lt;/ul&gt;

</content>
 </entry>
 
 <entry>
   <title>Building Infrastructure with Chef</title>
   <link href="https://thinkingonthinking.com/building-infrastructure/"/>
   <updated>2013-01-29T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/building-infrastructure</id>
   <content type="html">&lt;p&gt;Chef is a configuration tool for computing infrastructure, especially cloud infrastructure and helps to share infrastructure across projects, teams and products. What I like best on the Chef concept is, that it enables experiments with infrastucture, something wich otherwise would be costly or pointless, since building software and servers is often more of an art, as a science. For example, I remember browsing through a multitude of discussions in forum posts like “oh, I had to patch the lib-whatever header to make it work with version 3.2.x, not sure if this will work for version 3.1.y too”. So, in some senses, Chef is bringing defaults and shared efforts to infrastructure, similary to what Rails brought to building database driven web applications.&lt;/p&gt;

&lt;p&gt;As in most other successful open-source projects, Chef is heavily community driven in &lt;a href=&quot;http://lists.opscode.com/&quot;&gt;mailing lists&lt;/a&gt;, github repositories, &lt;a href=&quot;http://wiki.opscode.com&quot;&gt;a wiki&lt;/a&gt; and a vibrant IRC channel (#chef). I was very lucky to get in contact with &lt;a href=&quot;http://nathenharvey.com&quot;&gt;Nathen Harvey&lt;/a&gt;, who helped me a lot in understanding the basic configuration terminology that comes with Chef. Over the last months, we were able to discuss the basic steps to build servers from scratch (basically simple steps in “cooking” servers)&lt;/p&gt;

&lt;p&gt;So, what did I learn so far?&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;One of my most important insights so far is, that there is no ‘single’ Chef unlike a ‘single’ Rails for making web applications. What I mean is, that Chef comes in at least 2 flavors: Chef-Solo and Chef-Client (= Hosted-Chef/Chef-Server). Nathen discusses the basic difference between Chef-Solo and Chef-Server &lt;a href=&quot;http://nathenharvey.com/blog/2012/12/07/learning-chef-part-2/&quot;&gt;here&lt;/a&gt;. Now, some weeks later, my conclusion is that Chef-Solo is about managing a single node (sometimes for development purposes), whereas chef-server is closer to a multi-node, cluster-like server environment. Duh.. now it sounds logical, but in fact, it was rather confusing when I started learning Chef.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Chef-server comes with “knife”, a powerful command-line tool that helps in all kinds of steps for server configuration. Knife is that kind of tool that you start thinking of first, when creating/updating/extracing server build steps. Chef-Solo does not support knife as far as I know, but can easily be used together with hacks in a &lt;a href=&quot;http://docs.vagrantup.com/v1/docs/provisioners/chef_solo.html&quot;&gt;Vagrantfile&lt;/a&gt; or tools such as librarian-chef or berkshelf. Unfortunately, I haven’t played too much with those yet, although I wrote about an experiment with librarian-chef and Neo4J &lt;a href=&quot;http://thinkingonthinking.com/An-experiment-with-Vagrant-and-Neo4J/&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Server build steps are abstracted away in “cookbooks”. A cookbook is a bit similar to what a Gem is in a Ruby-on-Rails project. In fact, you can quickly get some server software running, just by including the cookbook, similar to a including e.g. a gem for authentication in Rails. However, as a single gem in a Rails project doesn’t make much sense, cookbooks are more powerful when coming in a bundle. So, a server is defined in terms of recipes in a cookbook, and it’s actually the “run_list” that tells Chef how a server should be built (or cooked).&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Ok, that is a summary of my Chef learnings so far. There are more insights related to how cookbooks work, how run lists can be simplified, and how bugs in cookbooks can be dealt with (tip: look in the github issues of a cookbook).&lt;/p&gt;

&lt;p&gt;In the meanwhile, I was able to work with cookbooks for a basic setup of Postgres and ArangoDB. And today, I just launched a basic web server instance on Amazon-EC2 with just a simple knife command.&lt;/p&gt;

&lt;p&gt;Last but not lwast, I got connected to some more DevOps, such as &lt;a href=&quot;https://github.com/jbianquetti&quot;&gt;Jorge Bianquetti&lt;/a&gt; who helped with librarian-chef and Cassiano Leal with whom I discussed &lt;a href=&quot;https://github.com/cassianoleal/vagrant-butcher/issues/3&quot;&gt;Vagrant-Butcher&lt;/a&gt;. Also, &lt;a href=&quot;http://mlafeldt.github.com/blog/2012/09/learning-chef/&quot;&gt;Mathias Lafeldt&lt;/a&gt; has some great discussions on his experiences on #learningchef. That’s all for now, looking forward to hear your feeedback, your insights, discoveries and problems with infrastructure automation. If you want to get more details on &lt;a href=&quot;http://nathenharvey.com/blog/2012/12/06/learning-chef-part-1/&quot;&gt;learning-chef&lt;/a&gt; check out the blog of &lt;a href=&quot;http://nathenharvey.com&quot;&gt;Nathen Harvey&lt;/a&gt; too.&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>What is a modular web application?</title>
   <link href="https://thinkingonthinking.com/what-is-modular-architecture/"/>
   <updated>2013-01-26T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/what-is-modular-architecture</id>
   <content type="html">&lt;p&gt;A number of patterns are driven by making systems modular, a concept that I read in &lt;a href=&quot;http://en.wikipedia.org/wiki/Object-Oriented_Software_Construction&quot;&gt;Object-Oriented-Sofatware Construction&lt;/a&gt; some years ago.&lt;/p&gt;

&lt;p&gt;Modules are abstractions to make software easier to write and to maintain, and ane of the most important pattern in application that faces users is called Model-View-Controller (MVC). A nice background discussion by Trygve Reenskaug (who discovered the MVC pattern in the 1970ies) can be found &lt;a href=&quot;https://groups.google.com/group/object-composition/msg/b9366f3bc78a33f8&quot;&gt;here&lt;/a&gt;, I will use the following definitions for MVC:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Model: All data items are representations of the real things; they make up the Model.&lt;/li&gt;
  &lt;li&gt;View: A View is componenent that transforms a model into something visible. A View is both input and output. A View is also a filter showing the interesting parts only [to a user].&lt;/li&gt;
  &lt;li&gt;Controller: The Controller sets up and coordinates one or more Views.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;However, in the context of (mobile) web applications, where views are living in (mobile) browsers that require a lot JavaScript, new design decisions for an application arise. Here are 2 questions that would impact the application design:&lt;/p&gt;

&lt;h2 id=&quot;1-how-do-we-serve-views-to-a-client&quot;&gt;1. How do we serve “views” to a client?&lt;/h2&gt;
&lt;hr /&gt;
&lt;p&gt;As long as a MVC stack lives on a server (e.g. a standard Rails application), this question is easy to answer. Views can easily access model instances, and the rendered views can be served from server to clients.&lt;/p&gt;

&lt;p&gt;In a mobile context however, the view rendering might happen on the client-side for efficiency. Or, in the context of a single-page application, views are only partially updated as needed. These designs require servers to prepare assets for clients, and new tool-chains are becoming popular. I wrote about some experiences of working with new tools for asset management &lt;a href=&quot;http://thinkingonthinking.com/MVC-and-Rails-API/&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Also, for serving frontends assets, no full application stack is needed. And since my programming background the last year was mainly in Ruby, I have been looking into serving assets with Sinatra and authenticating HTTP requests with Rack. Other basic assets servers, such as Express.JS and other Node.JS approaches might be interesting to look at too.&lt;/p&gt;

&lt;h2 id=&quot;2-how-to-organize-mvc-modules-on-the-client&quot;&gt;2. How to organize “MVC” modules on the client?&lt;/h2&gt;
&lt;hr /&gt;

&lt;p&gt;When working with MVC on the client, new decisions might be needed on how to organize an MVC structure. E.g. with Backbone, views are logic for state, events and accessing models, together with templates that help in rendering data.&lt;/p&gt;

&lt;p&gt;Additionally, there might be multiple MVC’s defined on a client, depending on the current scope of a HTTP conversation (e.g. /protected or /guest or /admin).&lt;/p&gt;

&lt;p&gt;So, far in Backbone, my application structure starts to look as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;/dashboard
  app.js
  main.js
  
  /collections
  /models
  /views
  /templates

/welcome
  app.js
  main.js
  
  /collections
  /models
  /views
  /templates
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

</content>
 </entry>
 
 <entry>
   <title>How I am (re-)discovering working with JavaScript</title>
   <link href="https://thinkingonthinking.com/how-I-start-liking-javascript-again/"/>
   <updated>2013-01-23T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/how-I-start-liking-javascript-again</id>
   <content type="html">&lt;p&gt;Having programmed in Assembler, C/C++, Java and in Ruby, I was always confused on how people are doing stuff in JavaScript - until recently luckily. In imperative languages, you learn to think about how to ‘control’ data; in JavaScript however, you think about this: What are your data structures (DOM/HTML/API resources)? And, how do changes in data structures look like?&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;As a start, the &lt;a href=&quot;http://www.json.org/&quot;&gt;JSON syntax&lt;/a&gt;, which is used everywhere in JavaScript, forces you to think about embedding and grouping values in data structures. Showing a resource in JSON can easily give feedback, if your data contains garbage (or not). And additionally, JSON in JavaScript code allows you to easily access attributes and properties. E.g. compare:&lt;/p&gt;

    &lt;p&gt;activity.friday.noon in JavaScript vs activity[:friday][:noon] in Ruby.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Next, the &lt;a href=&quot;http://underscorejs.org/&quot;&gt;underscore.js&lt;/a&gt; library brings a lot of Ruby-like syntactic sugar into working with JavaScript. Especially, it helps you to hide nasty, dynamic loops, that would require you to use incrementors (e.g. for (i = 0; … ) ) or suddenly break a loop. Underscore.js gives a lot of nice, little tools to work with lists and functions.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Your data structures can contain (anonymous) functions (= lambdas). This is great if you want to override default values with small computations when needed. Especially, using anonymous functions as properties is the core to understand how e.g. &lt;a href=&quot;http://backbonejs.org/&quot;&gt;BackboneJS&lt;/a&gt; enables a model-view-controller pattern (which is a rather complex topic, see &lt;a href=&quot;http://coding.smashingmagazine.com/2012/07/27/journey-through-the-javascript-mvc-jungle/&quot;&gt;http://coding.smashingmagazine.com/2012/07/27/journey-through-the-javascript-mvc-jungle/&lt;/a&gt;)&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Composition of functions is very easy with a library such as &lt;a href=&quot;http://requirejs.org/&quot;&gt;requirejs&lt;/a&gt;. This reminds me a bit on the concept of dependency injection, which in other languages is rather difficult to achieve (at least without getting lost). Anyway, there is also some discussion whether requiring JavaScript code with requirejs is actually &lt;a href=&quot;http://stackoverflow.com/questions/7708194/dependency-injection-with-requirejs&quot;&gt;injecting dependencies&lt;/a&gt;. Still, my experiments so far yield very, very modular code, something I like very much (getting out of the spaghetti code hell)&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;What are your experiences with JavaScript compared to other languages?&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>New kid on the Blog</title>
   <link href="https://thinkingonthinking.com/new-kid-on-the-blog/"/>
   <updated>2013-01-18T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/new-kid-on-the-blog</id>
   <content type="html">&lt;p&gt;The more I am getting into Ruby and Rails development, the more I experience that blogs help, inspire, connect, do business. So, it’s getting time to redefine my voice in the web, share memes on programming and web evolution. Maybe this post might inspire you or is helpful to you. Here is what inspired me and helped me:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;First, my motivation to share ideas in a blog was re-energized by meeting bloggers on conferences (BaRuCo, RuPy) and in the web, e.g. some include posts on Chef by &lt;a href=&quot;http://nathenharvey.com&quot;&gt;Nathen Harvey&lt;/a&gt;, Rails posts by &lt;a href=&quot;http://words.steveklabnik.com/&quot;&gt;Steve Klabnik&lt;/a&gt;, BDD posts by &lt;a href=&quot;http://blog.mattwynne.net/&quot;&gt;Matt Wynne&lt;/a&gt;, &lt;a href=&quot;http://businessmodelalchemist.com&quot;&gt;Alex Osterwalder&lt;/a&gt; on redefining business strategy, and last but not leasts product/software/devop posts by &lt;a href=&quot;http://svn.37signals.com&quot;&gt;37signals&lt;/a&gt;&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;When I restarted my blogging activities on blogger some weeks ago, I came across an article by &lt;a href=&quot;http://al3x.net/about.html&quot;&gt;Alex Payne&lt;/a&gt;, on how he ended up on hosting his blog on github pages. My last blogging activities were based on blogger.com; but programmers like editors, source control, and control of details in layouts and JavaScript interaction. So, time to say goodbye to blogger.com&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Somehow, I was reminded that in the &lt;a href=&quot;http://shop.oreilly.com/product/0636920019664.do&quot;&gt;Sinatra book&lt;/a&gt; by Konstantin Haase some Git/Sinatra workflow for blogging was mentioned. I tried this workflow, but Sinatra support on Github pages seems not to be the cutting edge. (If someone knows why, let me know). So, the cutting edge is &lt;a href=&quot;http://jekyllrb.com/&quot;&gt;Jekyll&lt;/a&gt;, a static site generator that “transforms your text into a monster”.&lt;/p&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Luckily, it took not much, to let Jekyll render the words you are reading here, from my editor to your web browser, because of 2 great web resources:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;The Jekyll parent blog of this blog came from &lt;a href=&quot;https://github.com/Sirupsen/sirupsen.github.com&quot;&gt;Sirupsen&lt;/a&gt;, who has put together a lot of nice design details into his templates.&lt;/li&gt;
  &lt;li&gt;A small overview on how to add all kinds of widgets to your blog, such as comments, Twitter links, and more: &lt;a href=&quot;http://erjjones.github.com/blog/How-I-built-my-blog-in-one-day/&quot;&gt;http://erjjones.github.com/blog/How-I-built-my-blog-in-one-day/&lt;/a&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Well, let me know what you think! I am happy to find out more about you readers, how you came hear, what you find helpful, when you read this. That’s all for now.&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>DOM feedback with JavaScript Jasmine</title>
   <link href="https://thinkingonthinking.com/DOM-feedback-with-JavaScript-Jasmine/"/>
   <updated>2012-12-18T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/DOM-feedback-with-JavaScript-Jasmine</id>
   <content type="html">&lt;p&gt;In my first attempts to develop frontend interactions with Backbone.js, I discovered some blind spots in my understanding of Backbone views. Especially, I find the different options to organize views with parent and child-views, as well as attaching views to DOM nodes difficult. A nice overview on Backbone pitfalls is given &lt;a href=&quot;http://blog.8thlight.com/cymen-vig/2012/12/13/reflections-on-using-backbone-js.html&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;As in many programming matters, thinking about outcomes and tests can help to write better code. This is where &lt;a href=&quot;http://jasminejs.org/&quot;&gt;Jasmine&lt;/a&gt; can help in the development of Backbone applications.&lt;/p&gt;

&lt;p&gt;First, there are several options to run Jasmine:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;The most common option for running Jasmine specs is by having your default web browser loading a test DOM (from e.g. spec/index.html or spec/SpecRunner.html). Within this option you can declare a number of JavaScript dependencies that contain the actual specs, and that will be executed as soon as the Jasmine execution is triggered (with jasmine.getEnv().execute(); ).
Another option is to run Jasmine specs through rake. This option seems to be very popular in the Rails community; but as I am thinking more towards development of stand-alone frontend applications, I don’t want to have a tight Rails coupling. I could not get rake jasmine to render my specs from the Backbone setup; so, I will postpone this approach for a while. Maybe someone of you has made some more successful experiments?&lt;/li&gt;
  &lt;li&gt;Another interesting attempt to get feedback from the DOM is to run Jasmine through a headless browser. PhantomJS comes naturally to mind in this context, and some interesting article is &lt;a href=&quot;http://blog.jphpsf.com/2012/10/31/running-Jasmine-tests-with-Phantom-js-or-Webdriver/http://obtiva.com/blog/112-javascript-specs-with-jasmine-a-primer-for-rubyists-part-1&quot;&gt;here&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;As a first step to testing a Backbone application, I wanted to test some basic Backbone view properties. Now, one of the difficulties that I found was, how to actually load all dependencies into a browser with SpecRunner.html including RequireJS; and, execute some specs.&lt;/p&gt;

&lt;p&gt;I found two approaches that work for me:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;
    &lt;h2 id=&quot;global-declaration-of-backbone-dependencies&quot;&gt;Global declaration of backbone dependencies&lt;/h2&gt;

    &lt;script type=&quot;text/javascript&quot; src=&quot;../public/js/libs/jquery/jquery.js&quot;&gt;

&lt;/script&gt;
  &lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;References to JQuery, Underscore and BackboneJS can be declared in a global way in the SpecRunner.html as follows:&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;This should give access to the Backbone library where you need them. It works good enough, but maybe it’s useful to load your dependencies more in a dynamic way if your application grows. For this, the next option might be more interesting.&lt;/li&gt;
&lt;/ul&gt;

&lt;ol&gt;
  &lt;li&gt;
    &lt;h2 id=&quot;loading-backbone-dependencies-with-requirejs&quot;&gt;Loading Backbone dependencies with RequireJS&lt;/h2&gt;
  &lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;When using several dependencies (including custom views, collections, routers, etc.), requiring every module in the global scope of SpecRunner.html might result into increased editing efforts. Here is an attempt to re-use some module definitions with requirejs that could in principle be shared between your tests and the real application.&lt;/p&gt;

&lt;p&gt;First, to load dependencies with requirejs, the following lines are needed in the specrunner.html&lt;/p&gt;

&lt;p&gt;The first line refers to the jasmine-require project by Scott Burch that helps to require dependencies from within specs. Some example how this can be used with a JavaScript framework is shown here from the Pattern JS project.&lt;/p&gt;

&lt;p&gt;With Backbone this might look like:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;  requireDependencies([&quot;underscore&quot;, &quot;backbone&quot;], function(_, Backbone) {
    View = Backbone.View.extend({tagName: &quot;li&quot; });
    view = new View();
  });
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;ol&gt;
  &lt;li&gt;
    &lt;h2 id=&quot;other-approaches&quot;&gt;Other approaches&lt;/h2&gt;
  &lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Another option to load dependencies with requirejs was described in &lt;a href=&quot;https://groups.google.com/forum/?hl=en_US&amp;amp;fromgroups=#!searchin/jasmine-js/requireJS/jasmine-js/7HJYbi705PE/XkpLR2DwQ0sJ&quot;&gt;this discussion&lt;/a&gt; in the Jasmine user group by using  testr.js However, I did not understand yet, how this is better than the previous option. (Maybe someone wants to comment?)&lt;/p&gt;

&lt;p&gt;Last but not least, there is an interesting Jasmine setup with RequireJS here by Peter Toomberg’s Shortcut project. This one does not require any additional setup declarations, but I did not yet look into this too much.&lt;/p&gt;

&lt;h2 id=&quot;conclusion&quot;&gt;Conclusion&lt;/h2&gt;

&lt;p&gt;Before actually running a succesful Jasmine spec for a Backbone view, we need a library to actually match DOM nodes with expected values. The common library for doing this are the Jasmine-JQuery matchers These allow to express many things, among if the .el property of a Backbone view actually match a DOM node as follows:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;View = Backbone.View.extend({tagName: &quot;li&quot; });
view = new View();


it(&quot;has el property&quot;, function() {
  expect(view.el).toBe(&quot;li&quot;);                                                                                                              
});
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;My current setup can be found here: &lt;a href=&quot;https://github.com/mulderp/backbone-require-test/tree/view_specs&quot;&gt;https://github.com/mulderp/backbone-require-test/tree/view_specs&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Well, so far my findings. Maybe they are helpful for others. I would be curious to hear what you think? How you approach testing of DOM nodes.&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>An experiment with Vagrant and Neo4J</title>
   <link href="https://thinkingonthinking.com/An-experiment-with-Vagrant-and-Neo4J/"/>
   <updated>2012-12-10T00:00:00+00:00</updated>
   <id>https://thinkingonthinking.com/An-experiment-with-Vagrant-and-Neo4J</id>
   <content type="html">&lt;p&gt;The RuPy conference 2012 in Brno was very inspiring! Especially, there were some interesting talks on databases and scalable approaches to web development:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;First,  the &lt;a href=&quot;http://www.arangodb.org&quot;&gt;ArangoDB&lt;/a&gt; team from Cologne set the tone why #nosql matters. The triAGENS team has written a database which mixes elements from MongoDB and graph databases. The shell of Arangodb looks very clean, and additionally, the system is based on C++ (= integrates with V8 JS engine and MRuby).&lt;/li&gt;
  &lt;li&gt;Another interesting talk was by Mitchell Hashimoto. He showed how Vagrant came into place, and why using isolated, virtual environments make sense for web integration. Some slides (not from Rupy) about this are &lt;a href=&quot;slideshare.net/mitchellh/sf-devops-introducing-vagrant&quot;&gt;here&lt;/a&gt;.&lt;/li&gt;
  &lt;li&gt;Andreas Ronge gave a very nice talk on what graphs can do, and SQL can’t (well, it can, but not nicely … ) I can’t find his slides from RuPy right now, but he maintains a great blog on Neo4J here:  &lt;a href=&quot;http://maxdemarzi.com/&quot;&gt;http://maxdemarzi.com/&lt;/a&gt;  Also, these slides are interesting: &lt;a href=&quot;http://slideshare.net/andreasronge/neo4jrb&quot;&gt;slideshare.net/andreasronge/neo4jrb&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So, ok, coming home to Munich with all these interesting thoughts in my mind, it was clear, that I had to start playing with graph databases in isolated environments for new kind of web applications. Fortunately, I had great input for my learnings from Jorge Bianquetti and Nathen Harvey.&lt;/p&gt;

&lt;p&gt;First about Vagrant and VirtualBox. It takes a bit of time to download virtual machines, but it’s not too difficult to get going. The single, most important command might be:&lt;/p&gt;

&lt;pre&gt;
$ vagrant init
&lt;/pre&gt;

&lt;p&gt;This creates an environment for setting up a virtual machine. It’s very cool, because now, you can imagine to setup an Ubuntu, Debian, CentOS, or whatever system, and vagrant will try to go ahead, download or copies the VM and prepares it just that you can use it.&lt;/p&gt;

&lt;p&gt;Ok, not quite, since next, you must tell Vagrant where to download the box; you do it in the Vagrantfile,  e.g.:&lt;/p&gt;

&lt;pre&gt;
config.vm.box = &quot;opscode-ubuntu-12.04&quot;
config.vm.box_url = &quot;https://opscode-vm.s3.amazonaws.com/vagrant/boxes/opscode-ubuntu-12.04.box&quot; 
&lt;/pre&gt;

&lt;p&gt;It’s the standard Ubuntu box from Opscode right now.
Just do a:&lt;/p&gt;

&lt;pre&gt;
$ vagrant up
&lt;/pre&gt;

&lt;p&gt;and you would have an Ubuntu box running.&lt;/p&gt;

&lt;p&gt;Well, we were charmed by graph databases, weren’t we?  Ok, so, let’s go ahead and add the setup for Neo4J. Some googling gives, that we have a Neo4J cookbook here: &lt;a href=&quot;https://github.com/michaelklishin/neo4j-server-chef-cookbook&quot;&gt;https://github.com/michaelklishin/neo4j-server-chef-cookbook&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Hmm.. in this stage, we actually decided already for chef-solo. There is chef-solo and chef-server, and if you want to understand the difference, I suggest you look &lt;a href=&quot;http://www.nathenharvey.com/blog/2012/12/07/learning-chef-part-2/&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Chef-server is the approach you want to use in production. Chef-solo is the approach for quick-and-dirty experiments, like we do here. So, let’s assume, chef-solo is ok, and we just need to get the cookbooks dependencies right. Luckily, we have a tool for this: chef-librarian.&lt;/p&gt;

&lt;pre&gt;
$ chef-librarian init
&lt;/pre&gt;

&lt;p&gt;This gives you a Cheffile. It’s similar to a Gemfile if you are used to Ruby.
Let’s throw in the Neo4J dependency here:&lt;/p&gt;

&lt;pre&gt;
cookbook &apos;apt&apos;
cookbook &apos;neo4j-server&apos;, :git =&amp;gt; &apos;http://github.com/michaelklishin/neo4j-server-chef-cookbook&apos;
&lt;/pre&gt;

&lt;p&gt;And now, similar to bundle install, we run librarian-chef install&lt;/p&gt;

&lt;p&gt;Last, but not least, we need to tell our VM that interaction with chef-solo is needed. You’ll do this by adding something into the Vagrantfile :&lt;/p&gt;

&lt;pre&gt;
config.vm.provision :chef_solo do |chef|
  chef.cookbooks_path = &quot;cookbooks&quot;
  chef.add_recipe &quot;apt&quot;
  chef.add_recipe &quot;neo4j-server::tarball&quot;
end
&lt;/pre&gt;

&lt;p&gt;Cool, now we only need to build our server, since our ingredients are prepared, and chef is ready for cooking. The magic command is:&lt;/p&gt;

&lt;pre&gt;
$ vagrant up
&lt;/pre&gt;

&lt;p&gt;You should see something like:&lt;/p&gt;

&lt;pre&gt;
[2012-12-10T20:33:58+00:00] INFO: *** Chef 10.14.4 ***
[2012-12-10T20:33:59+00:00] INFO: Setting the run_list to [&quot;recipe[apt]&quot;, &quot;recipe[neo4j-server::tarball]&quot;] from JSON
[2012-12-10T20:33:59+00:00] INFO: Run List is [recipe[apt], recipe[neo4j-server::tarball]]
[2012-12-10T20:33:59+00:00] INFO: Run List expands to [apt, neo4j-server::tarball]
[2012-12-10T20:33:59+00:00] INFO: Starting Chef Run for vagrant.vm
&lt;/pre&gt;

&lt;p&gt;This takes a while….&lt;/p&gt;

&lt;p&gt;Eventually, you end up successfully, and you can do:&lt;/p&gt;

&lt;pre&gt;
$ vagrant ssh
&lt;/pre&gt;

&lt;p&gt;Now, your VM has Neo4J running on it, and if you enable port forwarding in Vagrant, you might even go to: localhost:7474 and enjoy your fresh Neo4J server.&lt;/p&gt;

&lt;h2 id=&quot;some-references&quot;&gt;Some References:&lt;/h2&gt;

&lt;p&gt;Chef and Neo4J&lt;/p&gt;
&lt;ul&gt;
  &lt;li&gt;
    &lt;p&gt;Vimeo screencast: &lt;a href=&quot;https://vimeo.com/55085049&quot;&gt;https://vimeo.com/55085049&lt;/a&gt; , see: 
Screencast Install Neo4j-server with Chef from Patrick Mulder on Vimeo.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;Git repo: &lt;a href=&quot;https://github.com/mulderp/chef-neo4j&quot;&gt;https://github.com/mulderp/chef-neo4j&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;Chef basic steps with Nathen Harvey from Opscode
** &lt;a href=&quot;http://nathenharvey.com/blog/2012/12/06/learning-chef-part-1/&quot;&gt;Learning Chef Part 1&lt;/a&gt;
** &lt;a href=&quot;http://nathenharvey.com/blog/2012/12/07/learning-chef-part-2/&quot;&gt;Learning Chef Part 2&lt;/a&gt;
** &lt;a href=&quot;http://nathenharvey.com/blog/2012/12/14/learning-chef-part-3/&quot;&gt;Learning Chef Part 3&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</content>
 </entry>
 
</feed>
