Menu
Ok, with the app that I’m developing now, I know I need to learn how to do this.
You know when you’re just looking at somebody’s wall or something, on facebook and all of a sudden a comment just appears? I’m assuming this is because facebook pushes it to the browser and they don’t appear to do it with continuous server polling. How does facebook do this?
I understand they have a lot more resources than I am likely to have, so if I can’t do it the same way they do, how can I mimic this?
[CODE]
<!--[if lte IE 7]><script type='text/javascript' src="https://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js"></script><![endif]-->
<script>
function sideBarUpdate(id, cat, title){
//cache index for later or composite views:
App.cache.sidebar[id]=[cat,title];
//warm-up item page, cache in http pool for offline viewing:
new Image().src="/"+cat+"/article_"+id;
//grab all the article tags from the sidebar:
var barItems=( document.querySelectorAll || window.jQuery )("#sidebar .articles article");
//remove the last one on the list.
var last= barItems[barItems.length-1];
last.parentNode.removeChild(last);
//add the new one at the top:
var art=document.createElement("article");
art.innerHTML=title.link("/"+cat+"/article_"+id);
barItems[0].parentNode.insertBefore(art, barItems[0] );
}
</script>[/CODE]... what a waste of http requests and ferrying cookies around every 20 seconds. some of us are not on unlimited data plans or using battery cases...[/QUOTE]
While websockets [I]may[/I] be a better solution, if they were widely supported, it's a little silly to claim that long polling and other comet-like techniques are the sort of plague you're suggesting they are.
Now, consider a long-polling JSONP "connection" that re-establishes itself every 30 seconds over the primary domain, resubmitting all cookies (and other headers) with each re-connect. You're looking at about 600 bytes / reconnect, 1.2k/minute. To match the initial payload of yahoo.com in terms of "wasted requests", you need to be connected to the site for 500 minutes -- that's 8.3 hours.
So, you show up to work in the morning, connect to facebook, and if by the end of the day you've gotten no real data through the pipe, you've still only transferred 600kb of "overhead" to maintain your connection.
Not ideal, of course. But, certainly not the plague you're suggesting it is ...[/QUOTE]
im not saying they are the plague, im saying there are much better widely-supported solutions available.
2/3 visitors to our site at work arrive in websockets capable browsers. that's pretty wide, don't you think? 2 out of 3 ain't bad! sockets don't crash firebug if left open, which is nice.
the pages on this forum carry about 1.5kb of headers, and one should request every 20 seconds to avoid proxy timeouts, so X6 to your bandwidth calculations.
i was more concerned with battery life. 3g circuits stay on for 15 seconds after last activity, and burn a lot of battery while active. if you are spending 15 of every 20 seconds in active network state, your battery goes from standby life to 1.333X talk life to do nothing else...
i've used both, and i can say the while the user experience may be approximate with both, under the hood real sockets is much faster, lighter on both ends, and has few fewer side-effects than other comet techniques suffer such as cache stuffing, unwanted progress-bar activity, debugger net-tab overflow/shutdown, high mobile network utilization, and additional http traffic on the server. if you have 500 users connected and you send a comet update, you instantly have 500 incoming http requests. it's like a personal miniaturized DOS attack that will own poorly written DB code.[/QUOTE]
I was looking at websockets and socket.io, actually. What is the server architecture that is used for socket.io?
It looked like I could only do it with node...which brings me to another question...can I have a node server run alongside a regular apache server? How would node get the updated events - would the node server poll the database and push an event down the socket when it receives a change?
Sorry if I'm way off base here, I'm super new to this and I've realized I'm going to require information on not only the client but also the server side of things as well.
Thanks very much for both of your help so far.[/QUOTE]
Well no, I wouldn't call 66% "widely supported." For no other web technology would I make a recommendation based on 66%. For websockets, you can certainly make a case -- if you truly have a very low inter-visitor latency requirement, you can tell the other 33% to buzz off or that their experience will be less than great.
[/QUOTE]
I don't think you can bind to a port that's being use by another application. So, unless you're running your whole site with node, you can't use node for your websocket over port 80. But, you should just as easily be able to use port 8080 (or whatever) for your websocket connection.
In terms of the actual inter-user messaging, I think you can be creative here. I haven't written anything with node, so I'm being a little speculative here, but I think it's a stateful server application. You're effectively writing the web server itself (most of the work is done for you, of course). But, that means you can create pan-connection variables, message queues, etc. So, polling the database shouldn't generally be necessary -- not with a websocket server, and ideally not even with a good long-polling daemon.
The same would be true if you write a websocket server in PHP: you'd be writing an application that has a common state shared by all active connections. It's up to you to keep track of and ferry messages where they need to go however you can ... probably in a "shared" memory structure.[/QUOTE]
I was looking at websockets and socket.io, actually. What is the server architecture that is used for socket.io?
It looked like I could only do it with node...which brings me to another question...can I have a node server run alongside a regular apache server? How would node get the updated events - would the node server poll the database and push an event down the socket when it receives a change?
[/QUOTE]
[CODE]App.broadcast=function(strEventType, varData){
io.sockets.emit(strEventType||"error", varData || null);
};
// ...
fs.watchFile( path, function (curr, prev) {
if(curr.mtime-prev.mtime && curr.size >1 ){
App.broadcast("fileModified", path);
}//end if changed?
});//end watchFile callback
[/CODE]all the other comet techniques are a hack.[/QUOTE]
im suggesting that he use web sockets if available and fallback to xhr2 long-polling for IE, and jsonp long polling for anything else with unknown support. socket.io has a 5-method stack, but for hand-coded servers, the 3 mentioned are enough for 100% coverage.[/QUOTE]
if you have some data showing that long-polling and sockets have similer battery life, i'd love to see it. [B]The problem with polls on mobile is that they never let the phone sleep.[/B] With sockets, all the data arrives as it comes. there are no regular pings, and no extra connections per-event. [/QUOTE]
I'm simply reporting what i've found in an anecdotal fashion, and attempting to explain my observations with a hypothesis based upon the principles of cell communication as laid out by ATT. It's a lot of work setting up and tearing down an http connection compared to dripping a few bytes from the socket server.[/QUOTE]
Thanks, I'm just trying to get my head around this kind of programming. What I was thinking was to keep my current application served up through apache without modifications. 'Actions' performed by the user would be sent via routine ajax request to some php script on the server that updates something in the database. I had then considered using a websocket (or similar) to listen to a node server on 8080 that 'hears' this action (probably through database polling) and sends whoever is listening a notification.
Now that I just typed all that out, I'm thinking it would be easier if all actions that were performed were sent through node to the database rather than through a regular php script on apache - but I'm thinking that would require a major change in the backend of the application that I can't justify at this point.[/QUOTE]
OR ... you use the database primarily as a historical record. Perform all your message delivery with shared memory. [/QUOTE]
how do you do that in PHP, all the php chats ive seen use a DB or flat file to pass messages between client's back-end processes.[/QUOTE]
Both options are likely to be more efficient than hitting the database thousands of times per second -- even if you're only hitting a MEMORY table.[/QUOTE]
how do you do that in PHP, all the php chats ive seen use a DB or flat file to pass messages between client's back-end processes.[/QUOTE]
@svidgen :
i love you man, but you're being a negative nancy. i can tell you've not used socket.io because
1) you're wrong about implementation details
2) you're not singing it's praises from the mountain tops
i'll admit i was sceptical at first as well, but when i got to use it, I was like "OMMFG, i've been wanting this for years, and now its 2 lines of code!".[/QUOTE]
And probably the biggest and best case I can make against building any new app with websockets: I've seen no other major app that uses websockets -- all the big, successful companies still use long-polling. Consider Facebook and Google (gmail, docs, google "instant") -- apps that maintain more simultaneous connections than any other on the net without websockets. In fact, the only place I have seen websockets are on demo/experimental pages for web sockets![/QUOTE]
But, I couldn't justify the technology in a "real" application for another 3 to 6 years. My supervisor sure as hell wouldn't go for it -- even if I assured him that the 33% will be taken care of with fall-back mechanisms. He'd say, "No. We need to use a well-established solution for which we're familiar with the scalability, reliability, and security issues.[/QUOTE]
[CODE]npm update socket.io[/CODE]Sorry for hijacking your thread, aj_nsc ...[/QUOTE]
Well, I certainly agree that for operations wherein most browsers support *some* mechanism, a compatibility library is nice to have. But, I wouldn't depend on it for anything business critical.[/QUOTE]
i don't think sidebar updates are "business critical". isn't that taking it to 11 just a bit here? would you depend on js itself for anything "business critical"?[/QUOTE]
how do you do that in PHP, all the php chats ive seen use a DB or flat file to pass messages between client's back-end processes.[/QUOTE]
[code=php]<?php
// tell the driver script to send all necessary no-cache/expires headers
$do_not_cache = true;
// get ready to listen
$timeout = 30;
$starttime = time();
$channel = isset($_GET['channel']) ? substr($_GET['channel'], 0, 16) : "general";
$last_received = isset($_GET['last']) ? (int)$_GET['last'] : 0;
while ($starttime + $timeout > time()) {
$mq = apc_fetch($channel);
if ($mq && is_array($mq)) {
$rv = array();
foreach ($mq as $m) {
if ((int)$m['id'] > $last_received) {
$rv[] = $m;
}
}
if (sizeof($rv) > 0) {
$json_response = json_encode($rv);
return 1;
}
}
// wait 15ms before checking again
usleep(15000);
}
// timeout.
$json_response = "null";
?>
[/code][code=php]<?php
$do_not_cache = true;
$channel = isset($_GET['channel']) ? substr($_GET['channel'], 0, 16) : "general";
$message = isset($_GET['m']) ? $_GET['m'] : '';
// acquire next ID
if (!apc_exists($channel . "_message_id")) {
apc_store($channel . "_message_id", 1);
$m_id = 1;
} else {
$m_id = apc_inc($channel . "_message_id");
}
// acquire the queue
$mq = apc_fetch($channel);
if (!$mq || !is_array($mq)) {
$mq = array();
}
// purge old messages using an arbitrary max message value for now (128)
foreach ($mq as $k => $v) {
if ($v['id'] < $m_id - 128) {
unset($mq[$k]);
}
}
// add the new message
$mq[] = array(
'id' => $m_id,
'message' => $message
);
// store the new array in the cache.
// the TTL probably isn't necessary here ...
apc_store($channel, $mq, 30);
// return success ...
print "messageSent();";
?>
[/code]Oh, I'm just saying that right now is not the time to implement your server-interactions with websockets [or any other bleeding edge technology] or any 3rd party libraries that attempt to juggle variations in browser compatibility -- even if they appear to succeed at doing so. If that sort of thing was acceptable in a business setting, [I]successful businesses[/I] would be doing it. (And they're not.)[/QUOTE]
" any 3rd party libraries that attempt to juggle variations in browser compatibility" , you mean like jQuery? A lot of businesses use that for ajax, some of them even make money.
related, i don't think we should look for big business for idea on how to do web. 68% of fortune 500 companies use sharepoint, need i say more?
If you view the source of a website of just about any successful business, chances are you'll find terrible html, all sorts of proprietary meta tags and hacks, several analytical packages competing, etc.
google.com, apple.com, and microsoft.com don't even validate, and they are very successful. youtube is doing alright without IE6, even though the site is monetized no longer "universally compatible"...
my goal is to show the OP the simplest, most reliable way to get server-events up and running. socket.io ensures universal compatibility, optimum client machine utilization, and 1SLOC coding of actual event handling.[/QUOTE]
Hell ... I effectively just wrote an APC-based "chatroom" engine proof of concept in less than 15 minutes ... what could be simpler than that?[/QUOTE]
[CODE]
var http = require('http'),
url=require("url"),
bag={};
http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
var q=url.parse(req.url),
k=q.key,
v=q.value;
res.end( "cb(" + JSON.stringify( v ? (bag[k]=v) : bag[k] ) +")" );
}).listen(5555, '127.0.0.1');[/CODE][CODE]var http = require('http'),
url=require("url"),
bag={};
http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
var q=url.parse(req.url),
k=q.key,
v=q.value;
//assign v to message array:
if(v){ (bag[k]=bag[k]||[]).push(v); v=bag[k]; } else{ v=bag[k]||[]; }
//limit to 10 stacked messages
v.length=10;
res.end( "cb(" + JSON.stringify( v ) +")" );
}).listen(5555, '127.0.0.1');[/CODE]
So, let me be up-front here: I'm interested in how this works, if it does in fact work the same way as my posted scripts. However, if it doesn't provide the same functionality, bug-free, as my posted scripts, I can also claim to have written my messaging system in under 9 minutes (original write-up of both in about 5) ... before some simple optimizations and debugging.
Does it function like my script or not?[/QUOTE]
[CODE]//socket.io "chat room" demo
var App={
ip: "127.0.0.1" //change this to your server ip
},
http = require('http'),
url=require("url");
var PAGE=(function(){/*
<html><head>
<title>chat</title>
<style>
p, input { width: 80%; max-width: 50em; font: 14px monospace;
padding: 2px 4px;
background:#ddd; box-shadow: 1px 1px 3px #888; }
p { height: 12em; overflow: scroll; overflow-x: hidden; }
</style>
</head><body>
<input onchange="sendChat(this.value); this.value=''; this.focus(); "/>
<p>Waiting to connect...</p>
<button onclick="elm.innerHTML='';">clear</button>
<script src="/socket.io/socket.io.js"></script>
<script>
var socket=io.connect('/'),
elm= document.getElementsByTagName("p")[0] ;
function sendChat(txt){
log(txt.italics());
socket.emit("msg", txt);
}
function log(a){
log.r.unshift(new Date().toTimeString().split(/s/)[0].fixed()+ " - "+ a );
elm.innerHTML= log.r.join("n<br />");
if(!a.match(/<i/i)){ document.title="chat: "+a; }
return a;
} log.r=[];
socket.on('msg', log);
socket.on('connect', function (data) {
elm.innerHTML="";
console.log(log('connected.'));
});
</script>
</body>
</html>
*/}).toString().slice(15,-5);
//end HTML ###############################################
// create a server:
var myServer=http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/html'});
res.write( PAGE );
res.end();
}).listen(80, App.ip);
//socket.io stuff:
var io = require('socket.io').listen(myServer);
io.settings['log level']=1; //sshhhh...
io.sockets.on('connection', function (socket) {
socket.on('msg', function (data) {
socket.broadcast.emit( "msg", data);
});
});//end socket bindings
[/CODE]0.1.9 — BETA 6.17