Data Curation Experts

Developing Digital Repository Solutions

Spring Hydra Camp 2015 – Yale University Library

by Mark B.

We’ve just finalized details for the Spring 2015 Hydra Camp and registration is now open. As usual, the class provides an introduction to Ruby, Rails, the Hydra technology stack, metadata management using both XML and RDF data models, test driven development, and collaboration on open source software projects. With the production release of Fedora 4, we’ll also be showing off Sufia running on Fedora 4. This intensive four day course provides an overview of the skills and tools necessary to develop successful Hydra solutions once you return to your home institution.

Hydra Camp
Yale University Library
March 9th-12th, 2015
New Haven, CT
More Info + Registration

The course will run from 9am-5pm each of the four days: Monday, March 9th through Thursday, March 12th. The registration fee includes an optional Advanced Blacklight Workshop on Friday, March 13th. Standalone registration for the Blacklight workshop without HydraCamp is $100.

Advanced Blacklight Workshop
Yale University Library
March 13th, 2015
New Haven, CT
More Info + Registration

Topics for the Blacklight workshop will include:

  • Customized item-level views (by content type)
  • Overriding default behaviors and helpers
  • Supporting thumbnails
  • Running multiple catalog controller instances
  • Dynamically adding and removing catalog filters
  • Search field customization
  • Facet customization
  • Search across heterogeneous objects
  • Maximizing upgrade compatibility when customizing
  • Participant Q&A

If you plan to attend both events, please register for HydraCamp first to receive your link for complimentary registration for the Blacklight workshop. If you will only be attending the Blacklight workshop, please use the link above for stand-alone registration.

For more information about Hydra Camp in general along with a high level syllabus, visit our Hydra Camp information page.

Adopting Hydra in 2015?

by mplsalicia

New to Hydra? Wondering where to start? Here’s our perspective looking forward into 2015.

Which Fedora?

The content in a Hydra head is usually stored in Fedora. Most existing Hydra applications are running on Fedora 3.x. But if you’re starting fresh, we recommend using Fedora 4. The production release of Fedora 4 is out now, and it seems to be working well for the early adopters. The release candidate for Hydra running on Fedora 4 will be out within a week or two of this post. Rather than set yourself up for a major migration, we suggest you plunge right in with the latest and greatest.

Which gem?

One of the best things about adopting Hydra in 2015 is that you have options for jump-starting your project with gems. The Sufia gem is a well-tested, well-designed gem for a self-deposit Institutional Repository (IR). Sufia’s object model offers simple Dublin Core metadata attached to individual digital files. If you want a self-deposit system and Sufia’s file-focused object model and DC-based metadata meet your needs, we can’t recommend it enough.

If you want a self-deposit system that can handle more complex digital works comprised of multiple files (for example, a digital record album including files for each audio track, a track list, individual text files for lyrics, and an image file for the cover art), then Worthwhile is for you. Worthwhile is a streamlined IR gem with a minimalist philosophy. Originally based on the larger Curate project, Worthwhile pares away some of Curate’s more complex, less commonly needed functions to provide a basis that’s easier to adapt to a broad range of needs and environments. Despite its minimalist approach, Worthwhile provides the most frequently requested features found in both Sufia and Curate.

How to get bootstrapped?

There are lots of ways to learn about Hydra. You can work through community resources like the great online tutorials that focus on basic repository development. There are Hydra-specific events and Hydra-focused presentations at conferences and meetings throughout the year – the community keeps a continually updated list of events on the Hydra events calendar. If you haven’t already, we encourage you to join the one of the Hydra e-mail lists: either hydra-tech (developer focused) or hydra-users (more general topics and announcements). You can find instructions on how to sign up under the “mailing list” section of the Get-in-Touch page on the Hydra Wiki.

If you’re looking for help beyond community resources, DCE offers a range of services to help you bootstrap your Hydra project. DCE hosts at least two public Hydra Camps every year where developers can learn more about developing with Hydra in a supportive environment among peers (watch this blog for details about upcoming camps in Spring and Fall). For more specifics needs, DCE offers a variety of custom training, project planning, development, and co-development services to help get you and your team up to speed with Hydra. Please get in touch with us if you want to learn more about our offerings.

Welcome to the Community

The most important part of Hydra is the community. We want to make sure that people feel welcome and encouraged to participate. Are there barriers to entry? We may be too closely involved in the community to see them. We invite new adopters, new partners, and new developers especially to speak up on what the Hydra community can do to support you as you come on board. If everyone just takes what’s already done, and goes off to work on their own, the overall project will stagnate. If we all contribute, the project will flourish. So if you’re new to Hydra, post a comment and let us know what the community can do to help you connect.

Introducing Blacklight Folders

by mplsalicia

We’re pleased to announce the first release of the blacklight_folders gem, a plugin for Blacklight pioneered by Indiana University and built by Data Curation Experts.

Blacklight_folders is an enhanced reference organization tool. Authenticated users can:

  • create and manage multiple folders, private or public
  • save and order references within those folders
  • export folders with all their references as citations or to EndNote and RefWorks with blacklight_marc

Unauthenticated users have access to a default folder, to mirror the existing bookmarks functionality.

Blacklight_folders replaces the Bookmarks menu with a Folders menu. The code offers a migration script for existing bookmark records. The user interface is accessible to screen readers and supports keyboard navigation. It also includes a drag-and-drop interface for re-ordering references in a folder.

The repo is on github – we welcome contributors. For a limited time, DCE has a demo server running so you can see the code in action.

2014 Year in Review

by mplsalicia

In this season of thanksgiving and celebration, we’re looking back over 2014. From training to bespoke code to shared Hydra features to laying the groundwork for turn-key Hydra solutions, we spent the year closely involved in moving the Hydra project and the broader digital repositories conversation forward.

Here are some highlights from a very busy year:

Hydra Training and Outreach

  • We hosted 5 Hydra Camps this year including public camps at Yale University, Trinity College Dublin, and in our Minneapolis offices alongside two private camps to help Dartmouth College and the University of Alberta kick off major Hydra initiatives.
  • We spearheaded Dive into Hydra sessions at DLF, Open Repositories 2015, and both HydraConnect events.
  • As part of the HydraDAM project, we developed comprehensive installation instructions which can also be useful for other Sufia and Worthwhile based applications.
  • We led multiple public demos including a live demo of HydraDAM for the Association of Moving Image Archivists’ annual conference.
  • We helped plan and execute both Hydra Connect events this year and are looking forward to Hydra Connect #3 in Fall 2015.
  • Among the many institutions we helped introduce to the Hydra project, four institutions committed to becoming Hydra Partners this year: Tufts University, University of Alberta, Princeton University, and Case Western University.

New Gems & Projects

  • The worthwhile gem – a lightweight IR supporting complex digital objects (the object prototype for Hydra Works)
  • The blacklight_folders gem – a blacklight plugin for grouping and organizing bookmarks in folders
  • Batch ingest for Tufts MIRA admin head –  a building block for a more generalized batch solution in 2015
  • The tufts-image-library – an image head supporting ordered collections and slideshows inspired by Northwestern’s DIL project
  • Digital Case 2.0 – a full-featured IR based on the worthwhile gem
  • Hull History Centre – a blacklight search interface for cross-institutional collections, with EAD import capability

Additional Community Contributions

  • Work on fluctus, the admin interface for APTrust
  • Feature development for the initial release of Spotlight
  • ActiveFedora upgrades to support Fedora 4
  • Sufia upgrades to support Fedora 4
  • Lease & embargo functionality (developed for Digital Case 2.0, incorporated in core hydra codebase)
  • Hydra design and coding contributions at developers’ events in Palo Alto and Portland

Thanks to our clients and colleagues for a great 2014! We look forward to continuing our work and collaborations throughout the coming year.

Fall Hydra Camp 2014 – Princeton University Library

by Mark B.

We’ve just finalized details for the Fall 2014 Hydra Camp and registration is now open. The course will provide an introduction to Ruby, Rails, the Hydra technology stack, metadata management using both XML and RDF data models, test driven development, and collaboration on open source software projects. We are also excited that this will be our first camp to include a preview of Hydra using Fedora 4. This intensive four day course provides an overview of the skills and tools necessary to develop successful Hydra solutions once you return to your home institution.

Hydra Camp
Princeton University Library
August 26-29, 2014
Princeton, NJ
More Info + Registration

$50 – Early Bird Discount!
As a special incentive, we are offering a $50 discount on registrations made by July 15th, 2014 – just choose “Early Bird Registration” on the sign up page.

The course will run from 9am-5pm each of the four days: Tuesday, August 26th through Friday August 29th.  Friday afternoon is reserved for Q&A and student topics, so folks needing to leave early for Labor Day weekend travel can do so.

For more information about Hydra Camp in general along with a high level syllabus, visit our Hydra Camp information page.

Spring Hydra Camps Announced

by Mark B.

We’ve just scheduled two Hydra Camps for Spring 2014 – one in Minneapolis, MN and one in Dublin, Ireland!

Dublin, Ireland

April 8-11, 2014
Trinity College Dublin
More Info + Registration

Minneapolis, MN

May 6-9, 2014
Open Book Minneapolis (Downtown / Warehouse District)
More Info + Registration

For more information about Hydra Camp in general along with a high level syllabus, visit our Hydra Camp information page.

If neither of these classes fits your schedule, but you really want to attend an upcoming Hydra Camp, we’ll  be holding class on the East Coast in late September or early October 2014.  Check back in June for more details.

New Discount Structure

by Mark B.

At DCE, we are committed to the open source principles of transparency and sharing.  We’ve made a few changes to this year’s rate sheet which we hope embody these ideas.

  1. Any project we work on that is willing to license deliverables under an appropriate open source license will be eligible for our Open Source rates, which reflect an average 20% discount over our base work-for-hire rates. This reflects the reality that it’s actually more expensive for us not to share our work. Managing the overhead of tracking licensing on individual projects ends up costing us time and money that we would rather not have to charge for.
  2. Clients who choose to work with us on a strict Time-&-Materials basis will be eligible for a 10% discount. Committing to fixed cost projects involves a significant amount of risk and uncertainty for us, especially in light of the flexible and evolving requirements that govern most projects. We appreciate clients who actively engage in managing deliverables with us under a time-and-materials agreement. This is our way of recognizing their willingness to share the risks involved in producing high quality software.
  3. We are a small company and cash-flow is a concern for us. Waiting for payments to come through Accounts Payable departments sometimes gives us grey hairs. Clients who pre-pay (on smaller projects) or agree to net-5 terms (on larger projects) will be eligible for a 5% Early Payment Discount.

For Open Source work, this means that our base rates remain unchanged again this year.  In addition, we’re hopeful that a broader number of partners will be able to benefit from our new, simplified discount structure.

Download the full PDF: DCE Rate Sheet 2014 – Q1

The Hydra Production Stack

by Mark B.

I was at Case Western Reserve University last week leading HydraCamp.  Many of the folks attending were from institutions new to Hydra and eager to get repositories up and into production – and they had lots of questions about setting up Hydra in production environments.   As we progressed through the development content that the course covers, it became clear that the Hydra community hasn’t spent as much time describing the production operation of Hydra nearly as well as we have covered development topics.

Once you have a Rails development environment setup, the instructions in the Dive into Hydra tutorial will get a development environment up and running in less than a morning (as proven by our HydraCamp attendees!)  Our typical development environment  simplifies many setup and configuration decisions by relying on a handful of useful rake tasks and the installation of hydra-jetty which provides a pre-packaged java server with Solr and Fedora pre-installed and configured for development environments.

In a production environment, simplifying assumptions made for development environments aren’t appropriate.  When outside systems and support staff get involved in your project, they will want to understand all the parts involved.  As I looked around the developer and project wiki, I realized there wasn’t much that spoke to the large set of moving parts involved in a production environment.  So in the process of starting to try to answer the questions folks were asking, I thought I’d try to write down what I had floating around in my head so far:

ProductionStack v1.0 640x480

SYSTEM STACK represents the hardware, operating system, and core resources that systems and support folks often have strong opinions about, especially in regards to installation, patches, and operational support.

RAILS STACK items are things that are driven by Hydra, Rails, and your local development practices.  This part of your production stack is probably driven and governed by your development team as much or more than your production support team… unless you have a true DevOps model in place (if you do, we’d love to hear how you got there!)

OPTIONAL MODULES fill in the gaps between your core infrastructure and your Rails application. These are mostly linux utilities and rails gems that give you access to system tools and resources from within your rails apps.  They might include file characterization and rendering utilities and/or authentication and authorization libraries that help Rails talk to the rest of your environment.

Even this picture isn’t complete:  “Hardware” encompasses a lot of detail all by itself – disk and memory size, processor cores, network configurations, etc.;  “Bunder & Gemfile + Dependencies” encapsulates a lot of complexity; nearly every bullet point requires some time and effort to install correctly.  So, even though the slide isn’t exhaustive, it gave us a place to start talking.  And I think we began some good conversations.  We’ve been doing a lot of thinking at DCE about how to make Hydra deployment easier and more reliable and we plan to share more of our thoughts and ideas as they evolve over the coming months.

Welcome to DCE

by mhbussey

We’ve just launched Data Curation Experts (DCE) to meet the growing need for digital curation expertise and online repository management support. We are a global network of developers, archivists, designers, implementers, and others with broad experience in the digital repository and online archive fields. We help research institutions, museums and other organizations assess their digital repository and archive needs and implement flexible solutions to sustain their long term content needs.

One of the key motivations for launching DCE was our desire to build a flexible, scalable organization that can meet the needs of the growing global community of digital archivists and repository owners. Although we’re a very new organization, our members possess a wealth of experience in the field. Matt Zumwalt and Eddie Shin, founders of MediaShelf, are key instigators behind DCE. We’ll have lots more to share in the next few months, so check back soon!


Get every new post delivered to your Inbox.