desodorante
New member
David, Ekliptor,
This is an idea I have been thinking for a while but I am too bad at coding to explore, let alone implement.
First of all, I've been a follower of your work since Neomule (not sure if Ekliptor participated on that too) and so I am very excited for this project.
The problem I am looking to solve:
Provide new sources for dead files both in emule and BT (those that have no sources / seeds / peers)
The core idea is as follows:
Find a minimum, likely variable, micro-chunk size (likely byte sized) that can be exhanged between files of differesent sizes, hash and type.
The solution I am not able to explore:
Taking advantage of the P2P network interoperability of Neoloader, able to translate emule parts to BT pieces, and using the emule chunk hash system (9500 kilobyte chunks using the MD4 algorithm), identify smaller byte streams within the chunk that match other files regardless of final hash, size, or type.
Two approaches:
-While file is "healthy" (above 100% available), perform analysis of random parts against other file's parts to identify matching byte streams. Save this as a "map" for when the file is below 100%
-When file is no longer healthy (below 100%), complete missing parts with other random file's parts and hash until file checks out. Also save "map" once complete.
For instance, say FileA.avi is incomplete and had no sources for X time at the client side. Let Neoloader request a low queued file, download a chunk, and perform analysis in the client side. It could take the missing X bytes from the downloaded file for the incomplete chunk of FileA.exe and try to hash. If it fails, it could divide the size by 2, and take 2 streams, and repeat until a minimum size is reached.
The key to this feature is to find a reliable size to perform a search upon, and an algorithm to identify the similarity of 2 files in the network. I am confident there has to be a byte-sized stream to inspect that has a probability above chance to match other files, given the enormous ammount of information we have.
Of course this is likely going to be <64 bytes and meant for files that are nearing completion, I am thinking <1024kb left, not to swarm the networks with redundant requests.
If this works, Neoloader could have an ultimate killer feature: to revive dead files in P2P networks.
Also, this would mean you did not download a file, but a mesh of several, which has other implications.
Thank you and I hope you don't think I am crazy
Keep up the great work!
This is an idea I have been thinking for a while but I am too bad at coding to explore, let alone implement.
First of all, I've been a follower of your work since Neomule (not sure if Ekliptor participated on that too) and so I am very excited for this project.
The problem I am looking to solve:
Provide new sources for dead files both in emule and BT (those that have no sources / seeds / peers)
The core idea is as follows:
Find a minimum, likely variable, micro-chunk size (likely byte sized) that can be exhanged between files of differesent sizes, hash and type.
The solution I am not able to explore:
Taking advantage of the P2P network interoperability of Neoloader, able to translate emule parts to BT pieces, and using the emule chunk hash system (9500 kilobyte chunks using the MD4 algorithm), identify smaller byte streams within the chunk that match other files regardless of final hash, size, or type.
Two approaches:
-While file is "healthy" (above 100% available), perform analysis of random parts against other file's parts to identify matching byte streams. Save this as a "map" for when the file is below 100%
-When file is no longer healthy (below 100%), complete missing parts with other random file's parts and hash until file checks out. Also save "map" once complete.
For instance, say FileA.avi is incomplete and had no sources for X time at the client side. Let Neoloader request a low queued file, download a chunk, and perform analysis in the client side. It could take the missing X bytes from the downloaded file for the incomplete chunk of FileA.exe and try to hash. If it fails, it could divide the size by 2, and take 2 streams, and repeat until a minimum size is reached.
The key to this feature is to find a reliable size to perform a search upon, and an algorithm to identify the similarity of 2 files in the network. I am confident there has to be a byte-sized stream to inspect that has a probability above chance to match other files, given the enormous ammount of information we have.
Of course this is likely going to be <64 bytes and meant for files that are nearing completion, I am thinking <1024kb left, not to swarm the networks with redundant requests.
If this works, Neoloader could have an ultimate killer feature: to revive dead files in P2P networks.
Also, this would mean you did not download a file, but a mesh of several, which has other implications.
Thank you and I hope you don't think I am crazy
Keep up the great work!
