Data Curation Experts

Developing Digital Repository Solutions

Introducing Blacklight Folders

by mplsalicia

We’re pleased to announce the first release of the blacklight_folders gem, a plugin for Blacklight pioneered by Indiana University and built by Data Curation Experts.

Blacklight_folders is an enhanced reference organization tool. Authenticated users can:

  • create and manage multiple folders, private or public
  • save and order references within those folders
  • export folders with all their references as citations or to EndNote and RefWorks with blacklight_marc

Unauthenticated users have access to a default folder, to mirror the existing bookmarks functionality.

Blacklight_folders replaces the Bookmarks menu with a Folders menu. The code offers a migration script for existing bookmark records. The user interface is accessible to screen readers and supports keyboard navigation. It also includes a drag-and-drop interface for re-ordering references in a folder.

The repo is on github – we welcome contributors. For a limited time, DCE has a demo server running so you can see the code in action.

2014 Year in Review

by mplsalicia

In this season of thanksgiving and celebration, we’re looking back over 2014. From training to bespoke code to shared Hydra features to laying the groundwork for turn-key Hydra solutions, we spent the year closely involved in moving the Hydra project and the broader digital repositories conversation forward.

Here are some highlights from a very busy year:

Hydra Training and Outreach

  • We hosted 5 Hydra Camps this year including public camps at Yale University, Trinity College Dublin, and in our Minneapolis offices alongside two private camps to help Dartmouth College and the University of Alberta kick off major Hydra initiatives.
  • We spearheaded Dive into Hydra sessions at DLF, Open Repositories 2015, and both HydraConnect events.
  • As part of the HydraDAM project, we developed comprehensive installation instructions which can also be useful for other Sufia and Worthwhile based applications.
  • We led multiple public demos including a live demo of HydraDAM for the Association of Moving Image Archivists’ annual conference.
  • We helped plan and execute both Hydra Connect events this year and are looking forward to Hydra Connect #3 in Fall 2015.
  • Among the many institutions we helped introduce to the Hydra project, four institutions committed to becoming Hydra Partners this year: Tufts University, University of Alberta, Princeton University, and Case Western University.

New Gems & Projects

  • The worthwhile gem – a lightweight IR supporting complex digital objects (the object prototype for Hydra Works)
  • The blacklight_folders gem – a blacklight plugin for grouping and organizing bookmarks in folders
  • Batch ingest for Tufts MIRA admin head –  a building block for a more generalized batch solution in 2015
  • The tufts-image-library – an image head supporting ordered collections and slideshows inspired by Northwestern’s DIL project
  • Digital Case 2.0 – a full-featured IR based on the worthwhile gem
  • Hull History Centre – a blacklight search interface for cross-institutional collections, with EAD import capability

Additional Community Contributions

  • Work on fluctus, the admin interface for APTrust
  • Feature development for the initial release of Spotlight
  • ActiveFedora upgrades to support Fedora 4
  • Sufia upgrades to support Fedora 4
  • Lease & embargo functionality (developed for Digital Case 2.0, incorporated in core hydra codebase)
  • Hydra design and coding contributions at developers’ events in Palo Alto and Portland

Thanks to our clients and colleagues for a great 2014! We look forward to continuing our work and collaborations throughout the coming year.

Fall Hydra Camp 2014 – Princeton University Library

by Mark B.

We’ve just finalized details for the Fall 2014 Hydra Camp and registration is now open. The course will provide an introduction to Ruby, Rails, the Hydra technology stack, metadata management using both XML and RDF data models, test driven development, and collaboration on open source software projects. We are also excited that this will be our first camp to include a preview of Hydra using Fedora 4. This intensive four day course provides an overview of the skills and tools necessary to develop successful Hydra solutions once you return to your home institution.

Hydra Camp
Princeton University Library
August 26-29, 2014
Princeton, NJ
More Info + Registration

$50 – Early Bird Discount!
As a special incentive, we are offering a $50 discount on registrations made by July 15th, 2014 – just choose “Early Bird Registration” on the sign up page.

The course will run from 9am-5pm each of the four days: Tuesday, August 26th through Friday August 29th.  Friday afternoon is reserved for Q&A and student topics, so folks needing to leave early for Labor Day weekend travel can do so.

For more information about Hydra Camp in general along with a high level syllabus, visit our Hydra Camp information page.

Spring Hydra Camps Announced

by Mark B.

We’ve just scheduled two Hydra Camps for Spring 2014 – one in Minneapolis, MN and one in Dublin, Ireland!

Dublin, Ireland

April 8-11, 2014
Trinity College Dublin
More Info + Registration

Minneapolis, MN

May 6-9, 2014
Open Book Minneapolis (Downtown / Warehouse District)
More Info + Registration

For more information about Hydra Camp in general along with a high level syllabus, visit our Hydra Camp information page.

If neither of these classes fits your schedule, but you really want to attend an upcoming Hydra Camp, we’ll  be holding class on the East Coast in late September or early October 2014.  Check back in June for more details.

New Discount Structure

by Mark B.

At DCE, we are committed to the open source principles of transparency and sharing.  We’ve made a few changes to this year’s rate sheet which we hope embody these ideas.

  1. Any project we work on that is willing to license deliverables under an appropriate open source license will be eligible for our Open Source rates, which reflect an average 20% discount over our base work-for-hire rates. This reflects the reality that it’s actually more expensive for us not to share our work. Managing the overhead of tracking licensing on individual projects ends up costing us time and money that we would rather not have to charge for.
  2. Clients who choose to work with us on a strict Time-&-Materials basis will be eligible for a 10% discount. Committing to fixed cost projects involves a significant amount of risk and uncertainty for us, especially in light of the flexible and evolving requirements that govern most projects. We appreciate clients who actively engage in managing deliverables with us under a time-and-materials agreement. This is our way of recognizing their willingness to share the risks involved in producing high quality software.
  3. We are a small company and cash-flow is a concern for us. Waiting for payments to come through Accounts Payable departments sometimes gives us grey hairs. Clients who pre-pay (on smaller projects) or agree to net-5 terms (on larger projects) will be eligible for a 5% Early Payment Discount.

For Open Source work, this means that our base rates remain unchanged again this year.  In addition, we’re hopeful that a broader number of partners will be able to benefit from our new, simplified discount structure.

Download the full PDF: DCE Rate Sheet 2014 – Q1

The Hydra Production Stack

by Mark B.

I was at Case Western Reserve University last week leading HydraCamp.  Many of the folks attending were from institutions new to Hydra and eager to get repositories up and into production – and they had lots of questions about setting up Hydra in production environments.   As we progressed through the development content that the course covers, it became clear that the Hydra community hasn’t spent as much time describing the production operation of Hydra nearly as well as we have covered development topics.

Once you have a Rails development environment setup, the instructions in the Dive into Hydra tutorial will get a development environment up and running in less than a morning (as proven by our HydraCamp attendees!)  Our typical development environment  simplifies many setup and configuration decisions by relying on a handful of useful rake tasks and the installation of hydra-jetty which provides a pre-packaged java server with Solr and Fedora pre-installed and configured for development environments.

In a production environment, simplifying assumptions made for development environments aren’t appropriate.  When outside systems and support staff get involved in your project, they will want to understand all the parts involved.  As I looked around the developer and project wiki, I realized there wasn’t much that spoke to the large set of moving parts involved in a production environment.  So in the process of starting to try to answer the questions folks were asking, I thought I’d try to write down what I had floating around in my head so far:

ProductionStack v1.0 640x480

SYSTEM STACK represents the hardware, operating system, and core resources that systems and support folks often have strong opinions about, especially in regards to installation, patches, and operational support.

RAILS STACK items are things that are driven by Hydra, Rails, and your local development practices.  This part of your production stack is probably driven and governed by your development team as much or more than your production support team… unless you have a true DevOps model in place (if you do, we’d love to hear how you got there!)

OPTIONAL MODULES fill in the gaps between your core infrastructure and your Rails application. These are mostly linux utilities and rails gems that give you access to system tools and resources from within your rails apps.  They might include file characterization and rendering utilities and/or authentication and authorization libraries that help Rails talk to the rest of your environment.

Even this picture isn’t complete:  “Hardware” encompasses a lot of detail all by itself – disk and memory size, processor cores, network configurations, etc.;  “Bunder & Gemfile + Dependencies” encapsulates a lot of complexity; nearly every bullet point requires some time and effort to install correctly.  So, even though the slide isn’t exhaustive, it gave us a place to start talking.  And I think we began some good conversations.  We’ve been doing a lot of thinking at DCE about how to make Hydra deployment easier and more reliable and we plan to share more of our thoughts and ideas as they evolve over the coming months.

Welcome to DCE

by mhbussey

We’ve just launched Data Curation Experts (DCE) to meet the growing need for digital curation expertise and online repository management support. We are a global network of developers, archivists, designers, implementers, and others with broad experience in the digital repository and online archive fields. We help research institutions, museums and other organizations assess their digital repository and archive needs and implement flexible solutions to sustain their long term content needs.

One of the key motivations for launching DCE was our desire to build a flexible, scalable organization that can meet the needs of the growing global community of digital archivists and repository owners. Although we’re a very new organization, our members possess a wealth of experience in the field. Matt Zumwalt and Eddie Shin, founders of MediaShelf, are key instigators behind DCE. We’ll have lots more to share in the next few months, so check back soon!


Get every new post delivered to your Inbox.