Forensics tools and digital archiving

By paul on 29 June 2011 - 9:41am

I attended the DPC's latest briefing day on Digital Forensics for Preservation in Oxford on the 29th June 2011 and compiled a list of some of the software tools that were mentioned by the various speakers throughout the day.

The presenter who mentioned the tool is listed in brackets. A brief note on the purpose of the application follows the tool name. Note that this is focused on the presenter's use or reference in the context of their talk, and so may describe a particular function of a tool rather than provide a complete description of its purpose. Its also worth noting that I'm not an expert on digital forensics, so there may be some glaring errors here that I've not picked up on. The wifi at the event was a bit flakey so I wasn't able to do much online checking as I was writing up.

FTK Forensics Toolkit (John, Olsen)

  • integrated forensic toolkit
  • Now supports fuzzy hashing capability for identifying similar, if not identical, files

Attenex, Inspire, Greenstone Library (Attfield)

  • Visualisation tools

VI Threads (Attfield)

  • "Limited prototype" exploring the concept of visualising email conversations

Sleuthkit (John, Olsen, Knight)

  • Forensic analysis tools, UNIX based, open source
  • Not easy to use (Knight)
  • Autopsy (free), PTK (commercial) web client front ends for Sleuthkit
  • Autopsy can be quite slow, version 3 expected to address this issue
  • (see further notes in Gareth's comment below)

Hypatia (Olsen)

Muse (Olsen)

  • Email mining and visualisation (of Peter Koch's emails)
  • Prototype being developed, hope to release as open source software at some point in the future

Dc3dd (Knight)

  • imaging
  • booting from floppy

OSfClone, Guymager (Knight)

  • imaging
  • booting from CD/USB

OSForensics (Knight)

  • imaging
  • preferred by Knight
  • free

Digital Forensic Framework (Knight)

  • integrated forensic toolset

Pyflag (Knight)

  • integrated forensic toolset

OSForensic (Knight)

  • integrated forensic toolset
  • popular among archivists, easy to use (Knight)

Photorec, Scalpel, MagicRescue (Knight)

  • "Datacarving", extraction of files from images, deduplication
  • Mixed results, often many false positives

BulkExtractor/afflib (Woods)

  • Open source
  • Goal is to produce concise reports that support analysis/appraisal by non-technical staff
  • Indexes and supports analysis using a variety of means, occurence of words, email analysis, geo-location information, etc.
  • Identifies Credit card numbers, uses algorithms banks employ, rather than just looking for 16 digit numbers
  • Agnostic but not ignorant of underlying filesystem (supports several)
  • Focused more on digital archiving use case than some commercial tools
  • Runs a lot faster than many commercial tools such as FTK when indexing a drive, making it ideal for initial assessment to assist with targetting most interesting/relevant data
  • Will be incorporated into the Digital Curation Workbench
  • part of afflib.org that includes an imaging tool
  • Firewalk metadata crosswalk

Caine (Woods)

  • forensics toolkit

Some Other Random Notes From The Event

Published results of FIDO Project (which is currently evaluating forensic tools) will expand on Gareth Knight's talk.

Identifying typical system/software files with hash collisions (Knight), and matching against 3 possible libraries:

  • NIST National Software Reference Library (NSRL) hashset of legitimate files
  • Hashkeeper
  • Online File Signature Database (OFSDB)
    • NSRL most commonly used
    • Hashkeeper for law enforcement, can only gain access via FOI request
    • Useful also for identifying malware

Comment in questions following Knight presentation: Experiences of commercial imaging tools varied widely when imaging a damaged disk with bad sectors. Some performed well, some slow, some returned poor results. Comparitive testing, taking in free tools as well, might be useful here.

Comment in discussion: We need to check our content for viruses on access from our repository as well as on ingest, although old signatures are typically retired by Virus Checkers eventually. Alternative - provide access in a use once then throw away environment via VM constructed on the fly for access purposes.

Preservation Topics: 
Tools

Comments

garethknight - 4 July 2011 - 11:24pm

Hi Paul, thanks for the write-up. Just to clarify an aspect of my talk - it's the Autopsy front-end to SleuthKit that is quite slow to use.Sorry for any confusion caused. The Sleuthkit suite of tools are quite powerful and widely used within the OSS forensic world. There is, for example, an Sleuthkit and open source conference. The conference makes reference to an redeveloped version of Autopsy (3.0) that will hopefully resolve some of the issues with the current version of the user interface.

paul's picture

paul - 12 July 2011 - 1:01pm

Thanks for the correction Gareth. I think that was probably my misunderstanding! Looking forward to your FIDO reports! Almost forgot to say: I've tweaked the post to reflect this.