Let's do this thing! (15 Apr, 2019)

Have your say on today's Aardvark Daily column

Let's do this thing! (15 Apr, 2019)

Postby aardvark_admin » Mon Apr 15, 2019 9:43 am

This column is archived at: https://aardvark.co.nz/daily/2019/0415.shtml

Is this a project worth spending time and money on?

Will it be possible to get a team with the right skills and experience together to turn this concept into a reality?

What pitfalls do readers see that might lay ahead?

Given that one well-known Chinese-based company has already expressed a strong desire to work with me on a project like this, could we already be off to a good start?

How do you think Google, Facebook and governments might respond to this challenge to their dominance and authority?

Is it time we reminded ourselves (by way of a paradigm shift like this) exactly how the Net was built and how far we've drifted from the initial concepts of distributed content and multiply redundant pathways?

Anyone want to sign up for this bold venture?
aardvark_admin
Site Admin
 
Posts: 3542
Joined: Wed May 07, 2014 2:10 pm

Re: Let's do this thing! (15 Apr, 2019)

Postby Malcolm » Mon Apr 15, 2019 9:57 am

The dynamic DNS is already available, there are a bunch of commercial offerings, but my favourite is afraid.org. Set up a simple cron job on the Pi and it will update your chosen host name with the current WAN IP of the Pi.
Bigger issue is some providers are using carrier-grade NAT. So users on them won't get a real WAN IP. Not much of an issue in NZ at this stage. I think only Spark and Vodafone do it and only for 4G connections.
Of course the reliability and performance of the Pi will be the biggest catch. An SD card will wear out, low quality ones seem to be eaten by Raspberry Pis very quickly. And the performance of a Pi sitting on the wifi of a residential connection will cause issues, especially if you post a video which gains popularity. Finally how do you ensure no-one will be running out of date and vulnerable versions of Apache, MariaDB etc?
Malcolm
 
Posts: 345
Joined: Thu May 08, 2014 9:43 am

Re: Let's do this thing! (15 Apr, 2019)

Postby aardvark_admin » Mon Apr 15, 2019 10:32 am

The RPi setup would be only a stop-gap solution for early-adopters and low-traffic users that want to get started straight away. Interestingly enough, the microSD in my RPi which runs Kodi here (left going 24/7) has been in place for a couple of years low and still works fine.

The reality is that 10% of content creators create 90% of the traffic so those "elite" users would want something quite a bit more -upscaled than an RP but for the rest of the great-unwashed, the RP would be more than capable of serving up 2-3 concurrent HD video streams, even via its WiFi connnection.

As for keeping things up to date -- yes, that falls under the heading of "security" and is one of the key-factors in making sure everything is done *properly* right from the outset.
aardvark_admin
Site Admin
 
Posts: 3542
Joined: Wed May 07, 2014 2:10 pm

Re: Let's do this thing! (15 Apr, 2019)

Postby GSVNoFixedAbode » Mon Apr 15, 2019 11:50 am

Only real downside to this is the old Slashdot effect - if I'm hosting content on a local Pi and it becomes popular, my piece of string internet connection would get quickly overloaded. That's the only beauty of YT - bandwidth is their concern, and that's why we put up with it all.
GSVNoFixedAbode
 
Posts: 251
Joined: Thu May 08, 2014 8:53 am

Re: Let's do this thing! (15 Apr, 2019)

Postby aardvark_admin » Mon Apr 15, 2019 12:53 pm

I reckon you could build in some load-sharing for very popular videos. Have them automatically copied from busy servers to less-busy ones and let the search engines dynamically balance the load between them. Lots of challenges, lots of opportunities for smart solutions :-)
aardvark_admin
Site Admin
 
Posts: 3542
Joined: Wed May 07, 2014 2:10 pm

Re: Let's do this thing! (15 Apr, 2019)

Postby Malcolm » Mon Apr 15, 2019 2:22 pm

So, just put them all behind Cloudflare or Akamai? But then won't that just shift the control to the load balancing providers?
Malcolm
 
Posts: 345
Joined: Thu May 08, 2014 9:43 am

Re: Let's do this thing! (15 Apr, 2019)

Postby aardvark_admin » Mon Apr 15, 2019 2:28 pm

Malcolm wrote:So, just put them all behind Cloudflare or Akamai? But then won't that just shift the control to the load balancing providers?

I think it would be better to keep control of how the load-sharing was done. A server that detected a level of demand that was beyond its available resources (bandwidth, CPU, whatever) could send a request out to "known good" servers which had a surplus of the necessary resources and if one or more of those servers accepted the request, it could transfer a copy of the "in-demand" videos there with a "timeout" attached so that they'd automatically be deleted at some time in the future.

After that, the original server could, if necessary, simply act as a router for requests -- not actually serving up the video but passing the request on to the servers which it knew had a copy and the resources necessary to deliver that copy.

Keep it decentralised, keep it "neural" :-)
aardvark_admin
Site Admin
 
Posts: 3542
Joined: Wed May 07, 2014 2:10 pm

Re: Let's do this thing! (15 Apr, 2019)

Postby Muscular Jam » Mon Apr 15, 2019 6:05 pm

couldn't that be handled by existing opensource p2p software? What about hive2hive or InterPlanetary File System? I have a Pi and external drive doing nothing much.
Muscular Jam
 
Posts: 239
Joined: Wed Jun 21, 2017 4:09 pm

Re: Let's do this thing! (15 Apr, 2019)

Postby aardvark_admin » Mon Apr 15, 2019 7:26 pm

Possibly there is some P2P software that could provide key elements of the system -- that's something which needs to be considered so as to avoid reinventing the wheel.
aardvark_admin
Site Admin
 
Posts: 3542
Joined: Wed May 07, 2014 2:10 pm


Return to Today's column

Who is online

Users browsing this forum: No registered users and 4 guests