15 thoughts on “Metadata is not a “nice to have”

  1. Triparna Ray

    Interesting Matthew for me as I am from Bengal part of India who has red lentils almost every day 🙂 also as a power bi professional enjoyed the way you put forward the importance of metadata. Thanks!


  2. Love the analogy. My personal favorite analogy regards old family photos. If no one takes te time to write on the back of the photo the who/what/where/when/why (i.e. the metadata), that photo will get thrown away.
    As regards a dataset, where would you suggest that the metadata live? Is there a way to annotate it within the source or would it have to be a separate place (like an index is) where it lives?


    1. Unfortunately, there is no great general solution today. Enterprise data catalogs are often heavyweight and expensive. Ad hoc catalogs (and enterprise catalogs too, for that matter) are often disconnected from the experiences where people work with the data itself.

      In the ideal world, any data consumption or production experience will have capabilities for using and managing metadata as well, and those experiences would make it simple and easy for people to use and curate metadata as part of their core data tasks. But we both know we don’t live in that world today… ;-(


    2. Bad analogy. No one throws away old family photos because they lack who/what/where/when/why. In fact, I would argue that with family photos the metadata lives in the minds of the people in the photograph or some family member you haven’t yet spoken to.

      Somethings have value way beyond their metadata. I


      1. Jessica’s analogy is not a bad analogy – it’s just a different one, but Khurt’s response highlights a key part of the challenge.

        In far too many business contexts the metadata lives in the minds of the people who create and work with the data. It’s tribal knowledge, just like unlabeled family photographs. But as people move on to new jobs and the business changes over time, that tribal knowledge is lost. Even though the data may still be the same, and may still be valuable, when the people move on the tribal knowledge is lost. At this point it will either be organically rediscovered and recreated, or the data will stagnate because no one remembers anymore why it was important.

        I have personally seen this happen on multiple occasions with family photos, in the dark days following a funeral or a divorce.

        I think this will probably turn into a new blog post…


      2. New blog post please. 😃

        We lost my father-in-law to COVID-19 last week. It’s typical for Indians to collect and share photos of the deceased. Because of metadata on location/date, tagging of people and events, I was able to find and provide digital photographs from my personal catalogue. However, early film photographs of my father-in-law provided no context to my wife. But thank goodness we didn’t throw them out as the elders in the family were able to provide context and I was able to digitize those images.


      3. I’m very sorry for your loss. We’ll have that next post as soon as my schedule permits. I know that there are a lot of bored folks out there, but my team is busier than ever, and blogging has been moved to the back burner…


  3. My kind of post – metadata and a daal recipe. I’m thinking about Persistent Identifiers at the moment, and the same sort of principles apply. A name is very important of course, but without a persistent unique identifier it is very hard to make connections between that name and the things that name represents – publications, place of birth, occupations, aliases etc. However, like metadata, there are a whole host of options for what to use. Still, whatever you use, good structured metadata will help unite the saag paneers of the world!


    1. Thank you!!

      I absolutely love this idea, but it represents another anti-pattern for metadata: separating the data and the metadata. It’s undeniably a helpful workaround for a problem caused by the format in which the data was delivered, which means that the audience for the metadata will be reduced significantly.

      Quick segue:

      In the 80s I had a card catalog for my music collection. For each LP or cassette I would add a 3×5 index card with album and song details, and I would use this to manage the music and make it easier to find things… sometimes, sort of.

      But compare that today to a music experience like Spotify or iTunes (or whatever the cool kids use these days) where the data and metadata are delivered together, and the difference is like night and day.

      Although there are lots of things that we can do to work around a poor initial metadata implementation, they’re always just band-aids on a deeper problem. Treating metadata as an equal priority to data from day one yields a better integrated experience… but despite this, most software still treats metadata as an afterthought. ;-(


  4. Pingback: When does memory die? – BI Polar

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s