Comments on: lucene/solr meetup, july 28 /2010/07/lucenesolr-meetup-july-28/ Sarah Allen's reflections on internet software and other topics Tue, 10 Aug 2010 02:42:42 +0000 hourly 1 https://wordpress.org/?v=5.7.1 By: Our Solr system | Loggly /2010/07/lucenesolr-meetup-july-28/#comment-698 Tue, 10 Aug 2010 02:42:42 +0000 /?p=2753#comment-698 […] though its possible that we could lose data if things break. I clarified this a bit in a comment on Sarah Allen’s blog because I want to make sure the message is that 0MQ is awesome, not that it loses data. We […]

]]> By: Jon Gifford /2010/07/lucenesolr-meetup-july-28/#comment-697 Mon, 09 Aug 2010 21:17:54 +0000 /?p=2753#comment-697 Hi Sarah,

Thanks for the write-up, and apologies for the high-density talk :-)

I wanted to clarify one point in your writeup, though, to make sure people don’t get the wrong idea about 0MQ. Yes, our implementation of 0MQ has a potential “leak”, where we can lose messages, but its a very uncommon case, and the impact is small. Specifically, if one of the solr nodes dies hard, we potentially lose any events that were sent to it in the last batch (0MQ batches to minimize comms overhead). In steady state, 0MQ is rock solid, 100% reliable, and faaaaaast.

Pieter (at iMatrix) and I are currently discussing ways to solve the hard death problem, and I don’t anticipate it being a problem very long. As I said in the talk, 0MQ is unbelievably cool – if you haven’t got a project that needs it, make one up!

I’ll be posting on our blog in a day or so with the slides and some further clarification on a few of the things I (hurriedly) said in the talk.

Thanks again for the writeup,

Jon

]]>