tag:blogger.com,1999:blog-768233104244702633.post6728775844058870500..comments2023-11-16T03:16:54.746-08:00Comments on The Scale-Out Blog: Understanding Tungsten Replication ServicesRobert Hodgeshttp://www.blogger.com/profile/05379726998057344092noreply@blogger.comBlogger6125tag:blogger.com,1999:blog-768233104244702633.post-70047951031868980982014-04-14T13:37:30.122-07:002014-04-14T13:37:30.122-07:00@Moody,
Glad you liked the article. Slaves pull ...@Moody, <br />Glad you liked the article. Slaves pull from the master. They make a local copy of the THL records then apply that to the replica, generally using a JDBC connection in the case of relational DBMS like MySQL or using CSV in the case of data warehouses like Hadoop. <br /><br />Check out the replicator docs at https://docs.continuent.com/ for more information or post to the discussion list at https://docs.continuent.com/ if you have more questions.Robert Hodgeshttps://www.blogger.com/profile/05379726998057344092noreply@blogger.comtag:blogger.com,1999:blog-768233104244702633.post-56303040028771229542014-04-14T12:58:24.801-07:002014-04-14T12:58:24.801-07:00Hi Robert,
Thanks for this article, it helped.
...Hi Robert,<br /><br />Thanks for this article, it helped. <br /><br />I'm having some confusion on how the THL are applied to the slave(s). Does the tungsten manager push these events to the slave(s)? Or do the slaves pull?<br /><br />I am using MySQL btw and when I check the process list when dumping a file into a master, I am seeing a local connection (ex. slave1:35069). So it seems like tungsten is using an ssh tunnel to then apply through a localhost mysql connection? Just want to get clarification on the differences between tungsten and standard Mysql replication Hope this makes sense.<br /><br />Thanks,<br />MoodyAnonymoushttps://www.blogger.com/profile/18197590092041071894noreply@blogger.comtag:blogger.com,1999:blog-768233104244702633.post-34078067031364489702013-10-04T09:46:57.614-07:002013-10-04T09:46:57.614-07:00Thank you Robert for the explanation, it was inter...Thank you Robert for the explanation, it was interestingKaj Magnushttps://www.blogger.com/profile/18054221902017877054noreply@blogger.comtag:blogger.com,1999:blog-768233104244702633.post-77143333420930864612013-10-04T00:53:47.184-07:002013-10-04T00:53:47.184-07:00Hi Kaj, sorry about the ambiguity. Tungsten downl...Hi Kaj, sorry about the ambiguity. Tungsten downloads MySQL binlogs over the network to create local copies or reads directly from binlog files as they are written. Either way, we parse the binary format and turn it into our own log format, which we call the transaction history log (THL). <br /><br />As to your second question: we cannot use the binary logs directly for several reasons. <br /><br />(a) They only work for MySQL whereas Tungsten also reads logs other DBMS types like Oracle. <br /><br />(b) They lack important features like global transaction IDs, clearly demarcated transaction boundaries, column names as well as primary keys, and portable storage formats that translate easily to other DBMS data types. <br /><br />(c) Binlog formats are tied to particular MySQL versions--for example MySQL 5.6 binlogs are not compatible with 5.5 binlogs. <br /><br />As you can see there are many good reasons. :)Robert Hodgeshttps://www.blogger.com/profile/05379726998057344092noreply@blogger.comtag:blogger.com,1999:blog-768233104244702633.post-7365650354608901932013-10-03T15:44:13.164-07:002013-10-03T15:44:13.164-07:00(I forgot to subscribe to follow up comments; I...(I forgot to subscribe to follow up comments; I'm attempting to subscribe now, via this comment)<br /><br />Kaj Magnushttps://www.blogger.com/profile/18054221902017877054noreply@blogger.comtag:blogger.com,1999:blog-768233104244702633.post-38636737850885212182013-10-03T15:42:20.502-07:002013-10-03T15:42:20.502-07:00Concerning this: "Next we have Tungsten relay...Concerning this: "Next we have Tungsten relay logs. These are downloaded binlogs from a MySQL master DBMS from which the replication service creates the <i>Tungsten transaction logs</i>."<br /><br />I'm unsure about what "from which" refers to? Does Tungsten build <i>Tungsten transaction logs</i> from the relay logs ( = downloaded binlogs)? That is, does Tungsten look at the relay logs, parse them, do something with the parsed data, and the result is the <i>Tungsten transaction logs</i>?<br /><br />A follow up question: Why cannot the relay logs be used directly? Why are <i>Tungsten transaction logs</i> needed instead?<br />Kaj Magnushttps://www.blogger.com/profile/18054221902017877054noreply@blogger.com