One year ago...

One year ago I didn't know what WebRTC was. In fact, if someone would talked to me about a new in-development technology to directly connect web browsers between them and that I would became some months later a international reference doing keynotes and receiving several job offers thanks to it, I would give him a pokerface :-P

But just one year ago I had a dream. I dreamed about me asking myself why browsers couldn't be able to transfer files between them in real time and how WebSockets could help on this (real time communications between a browser and a server are the main purpose of WebSockets). After this and concluding it was a good idea, I asked myself "how the browsers would meet themselves? They would need an ID"... and at this moment I got something that would be comparable to a revelation or an epiphany: just came to my head an splash of the TeamViewer call ID window...

Just have an ID for each WebSocket connected to the server and use it as an address book! EUREKA!

After that, I was not able to get back to sleep :-P

So, I had the sensation of found something great, just great enough that if I got fired from my work (something that in fact it happened just less than a week after) it only would means that it was my destiny and I should as hard as possible to develop it. So, I asked for advice to a friend of mine about create something like PasteBin but for files (something that recently has developed Pipe). He was not confident about the commercial viability of the project (it would require both ends to be connected at the same time and also there's already a lot of ways to transfer files that doesn't have this requeriment, for instance DropBox), so I decided to take another way:

- Ok, so what if I convert it on a P2P network and register on the Universitary Free Software Championship?
- If you do it, you will crush the others and win them by several heads of advantage ;-)

So I did it... and several months later I won it :-)

I'm the one in the middle with the beard and the trophy in the hand :-)

To be honest, by that time I was not really sure about how I would do it (I've never been a networks guy) nor if someone had my idea before, so I started to look for some example code to start about. That's the way I ended finding DirtyShare, a proof-of-concept whoose concept was almost similar to the mine, but that was talking about something a new technology called WebRTC and also about the WebP2P maillist. That was interesting, so looking for it, I found that it was really a better approach that the original one using WebSockets, that it was starting to being available... and that the part where I was interested, the DataChannels, was not ready yet, so that was the reason I developed DataChannel-polyfill, people started to take interest on my work and I started to forge myself a name :-D

So, this has been a really long and intense year for ShareIt! and for myself, learning a lot and coding a lot and getting more success that what I would imagine on first place :-) But this doesn't means the end of this journey, since in fact I'm still working on it and I will show ShareIt! the next course as my degree thesis thanks to the collaboration of professor Luis López. That's the way seems that the version 2.0 is not only around the corner but also will be really awesome, with a high focus on portability and modularity, code reliability (WebP2P.io, the underlying handshake framework, it's a really nice piece of cake :-D ) and easyling to integrate it on other products, expanding it to new horizonts like using it for P2P audio streaming (yeah! :-D ). Thank you to everybody that has offered to me their support and please don't move from your seats or you would loose the fast moving news that are comming around ;-)

Achievement unlocked

Banned from W3C Restricted Media mail list for being critic with EME & DRM on HTML5. I love to be on the first line of fire... >:-D

And the winner is...

Two days late, but my mobile internet got down this weekend.

Just to say that ShareIt! has achieved the second position on the general category (first one was for Lynckia, that was using WebRTC too, by the way) and has won the Most Innovative Project category of the VII spanish national Free Software Universitary Championship 2013.

Yeah!!! :-D

Going mobile

One of the reasons why I though it would be cool to develop a P2P application inside the browser was that it could run everywhere there was a browser, without needing to install nor configure anything, just open a web page et voila! It was working. That's one of the reasons why some time ago I started to add some integration and responsive design to allow ShareIt! to work on mobile phones browsers and limited resources ones, not only natively but also as a "remote control" for a Node.js-based client, and it was fairly simple to modify the architecture and split the different components to make it possible, and also based on this architecture ShareIt! core could fully work inside a SharedWorker and being independent from the user interface (unluckily PeerConnections are not able to work inside Workers at this moment, but that's another story...). Problem was, I didn't have a mobile browsers modern enough to test it (also using WebSockets and my DataChannels-polyfill... mobile browser s*cks).

But now I have a Galaxy Note II, and can be able to run Chrome4Android for it :-) Since lately I was able to make ShareIt! work using native DataChannels, it was the perfect moment to give the mobile interface a new opportunity, and after some tune-ups on it, now I can say that ShareIt! is the first P2P application that can run on a mobile phone (and not just a proof-of-concept like the bittorrent port running on the Nokia S60 Python port... ;-) ). It need some improvements on usability and functionality (at this moment Chrome4Android doesn't support to share folders), but definitely it's almost as intuitively to use as the desktop version, and open new horizonts about what usages are possible with the WebRTC technology :-)

And if you are thinking about it... yes, DataChannels + reliable library are stable enough to be used on production, although is really slow (at least for me). I'm thinking about a solution for this like reducing packages size and wipe completely the need for reliable comunications, but I didn't take a final decision about this point...

Not so dead...

Remember my last post where I told would kill DataChannel-polyfill and work since now on native DataChannels because I got too much behind with them? Well...

...not so real :-)

Since the last post I have been thinking almost obsesively with the problem of needing a deprecated browser version just because of using my DataChannel-polyfill using WebSockets, and not working in latest versions with native support for DataChannels (that, by the way, sucks). So taking in account that the DataChannel-polyfill test code works with native ones I conclude that it would be just a problem on ShareIt! side not being fully compliant, so I went straight to it, removed all references to my polyfill and started again over with the handshake and PeerConnections layer from scratch, fixing one bug at a time.

And finally, after just some hours of work (instead of several days as I thought), it worked :-)

I must to admit that it's not the best implementation ever (I still ask my self why the h*ll is required to exchange some 'candidate' messages between the peers) and also it only support to transfer files and data up to 1Kb until they implement native reliable support (or I use reliable library, whatever comes first... :-) ), but the fact is that this almost mystical sensation after seen this thing working and with so few work (just some little tune-ups on the code thanks to the fact I develop DataChannel-polyfill trying to mimic the original specification API) it's really awesome :-D

So, not only I have experimental native DataChannels support on ShareIt! (F*ck yeah!!!!! :-D) but also being so little the diferences I've decided not only don't kill DataChannel-polyfill, but also try to improve it now that I know how native DataChannels work (and being so easy to use :-) ) and think about how to make a backward compatible API so older browsers could talk with newer ones :-)

But don't get me wrong little friends, the most important part of this post, if it was not clearly noticed yet, is the fact that now we don't need servers at all, being fully decentralized and serverless :-D Combine this with the Holy Grail of Internet that almost can bring us the HandshakeManager and the handshake servers, and wonder what an awesome future is waiting us... :-D

ShareIt! 1.1.1: Cloverfield

During the last weeks I was not able to publish too much here, specially because I'll start my exams the next week. But, since I got to be able to publish ShareIt! as my practice for my Networks-II class, I've been able to dedicate some time to it (maybe too much compared with the other asignatures... :-/ ) and develop some improvements directed by my tearcher Luis López, just in time not only for the lecture of the practice (tomorrow Monday on the morning :-P ) but also for the final phase of the championship, so here I'm showing to you the third public release of ShareIt! 1.1.1 "Cloverfield" (and no, although is the third release, the version is not written in unary ;-P)

It would be a fairly minor release because it only have two issues, but the fact is that it required some work on another ones and has added some great features and improvements on the architecture:

Manage duplicates: one of the problems that had ShareIt! previously is that since it was using the files hashes as unique identifiers of the files, if one of them was included twice only one of the copies whas showed (and served) also if it has different names. The solution was to identify them independently so both entries would coexist, and the ideal solution was to use the file's path. But what happens with the hashes? How I would be able to look for them if I'm not using them anymore as keys? I have been been putting off this thing a lot of time up to the point I didn't remember why I didn't do it before... and when I remembered it, I desired to forget it again.

IndexedDB: the reason why I have been been putting off so much the management of duplicates is because the optimal solution was to use IndexedDB's indexes, in fact a keyPath (multiple fields) for the primary key, and a non-unique index for the hashes (so they would be duplicated). What's the problem? Chrome hasn't still fixed the support of Blob and File objects inside IndexedDB (after almost one year and a half...), so I was alone using my polyfill. Two options: accept it, and told the reason to my teacher, or get mad myself, go for it and implement index support on my polyfill. At the worst, I would reuse later on my upcoming implementation of IndexedDB for Node.js for the headless version of ShareIt!... So I went directly for it, and after 8 hours listening to Daft Punk and the discography of Ke$ha and Demi Lovato (no comments), I got support for indexes on my own (in-browser, memory-only) pure Javascript IndexedDB implementation. #feelingLikeABoss... B-)

Sharedpoints: after that achievement another interesting point would have been detecting when a file has been added to a sharedpoint so it could be shared automatically without adding it by hand. Until this moment it was only able to detect when a file was deleted so it would be removed since I was using an input tag and its FileList object, that's mainly a photograph of the files that where available when the user selected it (although you can detect when it has beem removed checking an error when accessing to a file), and also it was done in a not maintable way, being all the management of the sharedpoints inside the hasher. Taking that in account, I decided to develop a new sharedpoint using DirectoryEntry from scratch, so I could be able to take in account the diferences and make it more portable. Now is fairly easier to add new types of sharedpoints like the ones I have been thinking about for Dropbox and Mega, and also I have been thinking about consider that internally the other peers would work as sharedpoints so I could simplify the way to access to them. Problem is, that I didn't notice until I finished it that Chrome has another bug not populating entries on the input tag and also I didn't be able to make drag&drop to work (this probably my fault...) so I've let this to be finished later.

And lastly by not least, the biggest feature added to this release and the one that made me think about tagging it as a mayor one, is the fact that now we have searches!!! :-D It's very primigenic and required some hacks (the fileslist is being shared between the peers after connection, having all of them the info of what are sharing others), but thanks to Lun-r.js now we can search by keywords, giving us a list of the files and hashes that correspond the them ready to be downloaded. It need some improvement in the future (like for example sending request to the others) but definitely it works :-)

But the reason why I didn't released it as a mayor release besides the fact that it has few (but important!!!) improvements, is that it will be the last of the 1.x series. This week I've discovered that ShareIt! doesn't work on Chrome browsers equal or newer than the current stable v26 because some API changes (the ones that almost crash the 1.1 release some weeks ago), so being only able to work on a legacy browser (Chrome v25) with my DataChannel-polyfill and without time in the next weeks to fix it because of the exams, I've decided to go directly to the 2.0 version after that and start using native DataChannels (I hope that by then reliable support will be implemented...), try to remove all the polyfills and develop all the backward incompatible features of the new protocol, like multiple and independent channels per PeerConnection on WebP2P or change the protocol to be RPC-based to increase reliability.

Good luck DataChannel-polyfill, and thank for the fish! :-D

WebP2P.io

I know that at this moment the WebP2P library is very tied to ShareIt!, specially because the universitary championship requires to have only one repository, but since I'll split it in the near future and in prevention of stealing it, I've decided to bought the (rather expensive...) domain WebP2P.io to host there the library project, focused in allow a server-less and distributed signaling channel for WebRTC applications. At this moment it only host a default webpage, but in the next weeks after I split the code I'll fill it with more info :-)