Last visit was: It is currently Sun Dec 09, 2018 11:39 pm


All times are UTC-05:00




Post new topic  Reply to topic  [ 3 posts ] 
Author Message
 Post subject: World biggest MD5/[other hash] database
PostPosted: Tue Nov 17, 2009 8:22 am 
External Project Staff
User avatar
 

Joined: Sun Oct 30, 2005 3:40 pm
Posts: 371
Website: http://idlesoft.net
Location: ~/
Well, how would you do it.

Requirements
Must be able to return an answer within a second or-so (so no rainbow tables)
Space efficiency is probably the most important factor, because these databases tend to be _HUGE_.

How I would go about it
Storing the MD5's on disk in a probably RB-tree index'ed way. Compression could helpful. I don't really know what more to say :classy:.

EDIT: Gheh, I described my first method in a way that would just store the MD5, no way to restore it (just what the md5 function does), now that was embarrassing :P

EDIT2:
More advanced solution, observing that MD5 is a hashing function, using this to build a DHT across a huge number of ever expanding nodes can be a good help. Still the same principle in the end though.

_________________
-- ChaosR

Image


Top
Offline   
 Post subject: Re: World biggest MD5/[other hash] database
PostPosted: Wed Nov 18, 2009 2:05 am 
Connoisseur of the Godawful
User avatar
 

Joined: Tue Mar 01, 2005 9:00 am
Posts: 456
ICQ: 286315965
Website: http://rabidtinker.mine.nu/
Yahoo Messenger: alistair_lynn
AOL: Agent_Vast@mac.com
Location: 127.0.0.1
DHT.

_________________
Alastair Lynn / Alumnus / Onlink Team


Top
Offline   
 Post subject: Re: World biggest MD5/[other hash] database
PostPosted: Wed Nov 18, 2009 1:41 pm 
Organ Donor
User avatar
 

Joined: Mon Aug 13, 2007 1:47 pm
Posts: 529
Location: Jawjuh
Quote:
DHT.
Or DMT. :sleazy:

_________________
Creative people must be stopped! (Latest Entry 7/31/11: "Fishsticks (18+))

Pleasantville by Night, a humorous horror web RPG


Top
Offline   
Display posts from previous:  Sort by  
Post new topic  Reply to topic

All times are UTC-05:00


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
Theme created by Miah with assistance from hyprnova