Speaker:
Dr. Ethan L. Miller
Professor, Computer Science Department
University of California, Santa Cruz
Schedule:
5th March, 2010
16:00 – 17:00
Place:
Tau Gallery, Tau Building
http://www.sfc.keio.ac.jp/en/campus_map.html
Title:
Distributed Metadata?and Indexing
Abstract:
The scale of today’s storage systems has made it increasingly difficult to find and manage files. To address this, we have developed Spyglass, a file metadata search system that is specially designed for large-scale storage systems. Using an optimized design guided by an analysis of real-world metadata traces and a user study, Spyglass allows fast, complex searches over file metadata to help users and administrators better understand and manage their files through the use of several novel metadata search techniques that exploit metadata search properties. Flexible index control is provided by an index partitioning mechanism that leverages namespace locality. Signature files are used to significantly reduce a query’s search space, improving performance and scalability. Snapshot-based metadata collection allows incremental crawling of only modified files. A novel index versioning mechanism provides both fast index updates and “back-in-time” search of metadata. An evaluation of our Spyglass prototype using our real-world, large-scale metadata traces shows search performance that is 1-4 orders of magnitude faster than existing solutions. The Spyglass index can quickly be updated and typically requires less than 0.1 of disk space. Additionally, metadata collection is up to 10 times faster than existing approaches.
This talk will also describe some recent results in extending the Spyglass work in several directions, including new approaches to index partitioning and new techniques to replace traditional file system index structures rather than simply augment them.
Biography:
Ethan L. Miller is a Professor in the Computer Science Department at the University of California, Santa Cruz, where he is the Site Director of the NSF Center for Research in Intelligent Storage (CRIS) and the Associate Director of the Storage Systems Research Center(SSRC). He received an Sc.B. from Brown University in 1987 and a Ph.D. from UC Berkeley in 1995. His current research projects include long-term archival storage, scalable metadata systems, file systems for non-volatile memory technologies, reliable and secure storage systems, and issues in petabyte-scale storage systems. Prof. Miller led the team that developed Pergamum, a brick-based long-term archival storage system, and was a member of the group that developed Ceph, a high-performance petabyte-scale distributed file system. Prof. Miller’s broader interests include file systems, operating systems, parallel and distributed systems, information retrieval, and computer security.