Managing unstructured data

Why is it important to manage “unstructured data” in your organisation ?

Enterprise Info managementIf your organisation produces emails, documents, client records, social media, audio, video; then you definitely need a plan for how you are going to manage, store and retrieve this information.

What are the differences between structured and unstructured data ?

Structured data is usually stored in a database, in labeled information fields and is easily searched for using a query i.e. a customer information system in which you can search by customer name or number.

Unstructured data may have a structure such as an email, but the information contained in the email has no structure, and is difficult to find unless there is a method for indexing and storing this information.

Examples of “unstructured data” are:

  • Emails
  • Books
  • Documents
  • Journals
  • Paper based information or records
  • Audio recording such as that from a Call Centre phone logs
  • Video
  • Blogs, Tweets, SMS text messages, faxes

What are some reasons for needing to store and retrieve and analyse unstructured data?

  • To answer a media, ministerial or management query.
  • To analyse the performance of your organisation or business and make better decisions.
  • Legal compliance
  • Customer complaint handling
  • Quality management
  • To make decision on pricing, staffing, location of branches or  offices or outlets

What should a plan for managing unstructured data cover?

  • An information lifecycle strategy and roadmap.
  • A recordkeeping strategy and roadmap (what information should you store and what to discard.
  • Plus how long should you store it?)
  • Details on how to integrate the new approach into your organisation and make it operable.
  • Identify data sources and repositories.
  • Set out storage needs and strategies.
  • Define formats, indexes and language (taxonomy)
  • Enable compliant retention and disposal in systems
  • Plan for migration of data or content to new solutions.
  • Strategies for data access using techniques and tools for data mining and analytics.

Be aware of the relevant NZ and international standards

Relevant international standards are to be found at:

  • International Organization for Standardization (ISO)
  • The International Congress of Archives (ICA)
  • The European Model Requirements for the Management of Electronic Records (MoReq)
  • The United States Department of Defence 5015.2-STD Design Criteria Standard for Electronic Records Management Software Applications (DoD 5015.2).

New Zealand standards are:

  • Archives NZ Digital Recordkeeping Systems (DRS) standard
  • (itself an adoption of the following ICA/ISO standards)
  • ISO TR 23081-3 Information and documentation — Managing metadata for records — Part 3: Self-assessment method.
  • “N800R1 Where to start – advice on creating a metadata schema or application profile v8 Functions”
  • ISO 15489 Information and documentation – Records management.

Use Knoware’s experienced team to make progress in managing and using unstructured data.

  • We have worked on developing and using the ISO standards
  • Our team has presented papers on this subject for example at the International Council on Archives Congress and the NZ Computer Society.
  • We work with NZ Archives and understand the direction that government is heading towards.
  • Our analytics and data mining specialists are experienced in helping organisations find answers using tools and techniques for both unstructured and structured information.