2 headed snake logo
libraries ☞ minimal computing ☞ soviet sci-fi ☞ fiber art.

This post is part 4 of 4 in a series. Feel free to skip around to:

part 1: the task, part 2: data transformation, or part 3: the site.


The demo!

The mostly finished demo has directories of plays, productions, performances, authors, performers, characters, kashira, scenes, and image tags

with individual layouts displaying and linking object data together.

It is navigable through the above directory listings, through several dynamic search boxes running client-side Lunrjs, and via clickable D3js data visualizations. It handles relative/massive image sets by implementing lazy load in a jQuery carousel.

To learn more about implementing D3 visualizations in Jekyll, you can check out this post. And a post on building multi-language Lunr indexes in Jekyll should be coming soon!


   Started with a Cake PHP site powered by a relational MYSQL database.

   MySQL dump to CSVs.

   Imported CSVs into IPython as Pandas dataframes.

   Merged relational data (from CSV jointables) onto dataframes by type.

   Exported dataframes as JSON records (and CSVs, for archival purposes only).

   Dropped null key:value pairs from JSON using JQ.

   Generated Jekyll collections (and pages) from YAML using wax_tasks gem.

   Ended with a ~40k page static Jekyll site powered by YAML data, with JSON index for client-side search.

The Summer of Japanese Puppets, Part 3 Peak Laziness: Automate docs for DB updates with Pandas + Markdown