Reducing Rousseau - Dictionary making with the Shell

by Hopton, Paul 4 March 2010 in MMS Software

Jean Jaques Rousseau

This evening I needed to get a list of unique words for a captcha tool. I decided to start with one of my favourite books of all time: 'Confessions' by Jean-Jaques Rousseau. I grabbed the text courtesy of the Gutenburg project and then had to figure how to extract the words. Rousseau was not only an eloquent writer but posessed a wide vocabulary (even when translated into english).

I googled for some scripts, but then just decided to pipe stuff together on the command line. here's how it looked:

grep -o '[a-zA-Z]\{4,12\}' rousseau.txt | 
    tr A-Z a-z | 
    sort -u |
    tr '\n' ';' > rousseauWords.txt

this processed the whole book in less than 2 seconds and gave me a list of all uniques words longer than 3 characters and shorter than 13, stripped of punctuation, turned to lowercase and seperated by a colon.

Not particularly eloquent, and I'm sure there is some repetition but it does the job nicely

Tags: geek and shell

Share |

You can comment on the post using the form below. You can also Trackback from your own site. If you would like to follow Comments on this Post you can follow feeds on Atom, RSS and RSS2

< 1 | 2 | 3 | 4 | 5 | 6 >

Tags

less rain, red bull soapbox racer, awards, berlin, graffiti, talk about the weather, blog features, japan, art, electric bike, falling walls, flash, street art, tokyo, video, Einstein Foundation Berlin, design, event, game, iPhone, product design, vandalsquad, walton ford, CKEditor, FWA, Meyer miller smith, animation, architecture, branding, corporate identity, designtide, exhibition, identity, interface design, interior, interior design, lars eberle, london, motion graphics, red bull flugtag flight lab, table berlin, tools, webby, 3d, AU, Baden-Württemberg, Brose, DRY, David Ajaye, EnBW, Fahrer-Assistenz-Systeme, Figaro, Gadgets, HTML5, Hotel-Tipp, Integration, Interactice TV, Internet, Kruisherenhotel Maastricht, MediaMix, Mee-Mail, Meyer, Miller, Motivation, Nussknacker, Porsche 911 Turbo S, Quadricopter, Roding Automobile, Sachsenring, Sagmeister, Saul Bass, Shekhar Kapur, Smith, Swarovski, Test, They thought we wouldn't notice, Twitter, XXX, Zukunft, adventure, advertising, akari luig, application, as if film matters, as3, auto, bbc, book release, car, carnivores, character design, christmas, cinematography, clios, conference, configuration, craft, creative industry, dadako, deer vase, destination death, documentary, doom, eco2, eelectric bike, energy, error handling, expo, film Noir, first-person, fitc, flasg, food, funny, furniture, geek, google api maven, green browsing, gunnar bauer, hamburger bahnhof, hawken, holger int'veld, html, human rights, i want one of these, illustration, installation, interactive, interesting, internet case study book, japanese, jason koxvold, java, javascript, jewelery, jules shulman, learning, less colour, logo, logo design, machinarium, madness, manner poster, marble, maria volokhova, marketing, martin andersen, metro, michael hecken, mindkiss, mobile Gesellschaft, mobile manual, mootools, movie, mozart, music, music video, nalden., naming, niels Meulman, ok go, one show, online advertising, online game, oubey, packaging, paper craft, paris, philip hennevogl, photography, politics, print design, prototype, quatschtronauten, ralf bähren, random things, red bull, ronzo, rube goldberg, schlachtfest, sculpture, self initiated, shell, shoe, slow motion, software, south africa, spring, startup, stop motion, surf, taschen, tetris, tha ltd, tiwiki, tokyo clash, tshirt, viral, visual acoustics, wikipedia, zipper

Meyer, Miller, Smith.

Head. Hands. Heart. We believe that there is a real connection between craftsmanship and communication. Working together as a group of skilled craftsmen we fuse contemporary design (Meyer) and the art of storytelling (Miller) with cutting edge technology (Smith), relentlessly pushing into new territory by creating pioneering work.

As information and intelligence becomes the domain of computers, society will place more value on the one human ability that cannot be automated: Emotions.

People want to experience beauty, enjoy one’s work, feel passion, they want to interact with each other. We all want.

That’s why we believe that the future of brands is interaction, not commodity. It’s not something you buy, but something you participate in.