What are the differences between structured and unstructured data?
Structured data is usually stored in a database, in labeled information fields and is easily searched for using a query i.e. a customer information system in which you can search by customer name or number.
Unstructured data may have a structure such as an email, but the information contained in the email has no structure, and is difficult to find unless there is a method for indexing and storing this information.
Examples of “unstructured data” are:
- Emails
- Books
- Documents
- Journals
- Paper based information or records
- Audio recording such as that from a Call Centre phone logs
- Video
- Blogs, Tweets, SMS text messages, faxes
What are some reasons for needing to store and retrieve and analyse unstructured data?
- To answer a media, ministerial or management query.
- To analyse the performance of your organisation or business and make better decisions.
- Legal compliance
- Customer complaint handling
- Quality management
- To make decision on pricing, staffing, location of branches or offices or outlets
What should a plan for managing unstructured data cover?
- An information lifecycle strategy and roadmap.
- A recordkeeping strategy and roadmap (what information should you store and what to discard.
- Plus how long should you store it?
- Details on how to integrate the new approach into your organisation and make it operable.
- Identify data sources and repositories.
- Set out storage needs and strategies.
- Define formats, indexes and language (taxonomy)
- Enable compliant retention and disposal in systems
- Plan for migration of data or content to new solutions.
- Strategies for data access using techniques and tools for data mining and analytics.
Be aware of the relevant NZ and international standards
Relevant international standards are to be found at:
- International Organization for Standardization (ISO)
- The International Congress of Archives (ICA)
- The European Model Requirements for the Management of Electronic Records (MoReq)
- The United States Department of Defence 5015.2-STD Design Criteria Standard for Electronic Records Management Software Applications (DoD 5015.2).
New Zealand standards are:
- Archives NZ Digital Recordkeeping Systems (DRS) standard(itself an adoption of the following ICA/ISO standards)
- ISO TR 23081-3 Information and documentation — Managing metadata for records — Part 3: Self-assessment method.
- “N800R1 Where to start – advice on creating a metadata schema or application profile v8 Functions”
- ISO 15489 Information and documentation – Records management.
Use Knoware’s experienced team to make progress in managing and using unstructured data.
We have worked on developing and using the ISO standards.
Our team has presented papers on this subject for example at the International Council on Archives Congress and the NZ Computer Society.
We work with NZ Archives and understand the direction that government is heading towards.
Our analytics and data mining specialists are experienced in helping organisations find answers using tools and techniques for both unstructured and structured information.