cypherpunks logo cypherspace logo datahaven logo

Design alternatives

Consider we have a number of internet nodes serving fixed size (say 10k) blobs of encrypted data via http. Each blobs has a unique blob-id.

There seem to me to be 3 basic approaches to locating information:

  1. distribute the data according to blob-id prefix. Number the servers 0..n-1, place blob-ids at server (blob-id mod n). Have some redundancy so there are multiple servers with the same server-id. Then locating data is simple, just pick a server at random from the list of servers sharing server-id.

    Apparently some smart caches and high performance proxies do something like this based on document hash.

  2. Have a URL server which maps virtual document URLs to server-id and blob-ids. Either user distributes data as they see fit, or servers self organise and keep random data, or use LRU cache replacement policy.

    Ian Goldberg & David Wagner's TAZ does something like this -- the TAZ is a virtual URL database. (Their rewebber is chained encrypted URLs).

  3. Have a web of servers which will request information from each other, where the web is organised however the operators see fit, or using some agreed network structure. You make a request from any node and it returns it if it has it

    Freenet proposes to do something like this. Not much activity on freenet project at present.

Obviously there are lots of design permutations that could be tried.