tag:blogger.com,1999:blog-768233104244702633.post8027042600217249447..comments2023-11-16T03:16:54.746-08:00Comments on The Scale-Out Blog: MySQL to Vertica Replication, Part 2: Setup and OperationRobert Hodgeshttp://www.blogger.com/profile/05379726998057344092noreply@blogger.comBlogger13125tag:blogger.com,1999:blog-768233104244702633.post-60912925197970584752014-10-20T08:02:14.744-07:002014-10-20T08:02:14.744-07:00Thank you Robert for a very good article over setu...Thank you Robert for a very good article over setup of mysql-vertica replication. I have run into broken replication at tungsten slave, while the bin logs at Mysql were dropped through auto purge. <br /><br />Is there a way to fix the replication at slave replicator and eventually to vertica, some how like initiating a resync operation? Anonymoushttps://www.blogger.com/profile/13093680772764774076noreply@blogger.comtag:blogger.com,1999:blog-768233104244702633.post-89816838424602184782013-04-11T15:40:34.727-07:002013-04-11T15:40:34.727-07:00@Robert
I'm using vertica 6.0.1. I saw the d...@Robert<br /><br />I'm using vertica 6.0.1. I saw the docs on the official site, but found this blog easier to follow. I think compared to the official docs, I'm just adding --home-directory on my slave config and not using the vertica6 batch-load-template because the release version doesn't include that yet.Anonymoushttps://www.blogger.com/profile/04495553806374188221noreply@blogger.comtag:blogger.com,1999:blog-768233104244702633.post-69402479452614164172013-04-10T22:10:44.203-07:002013-04-10T22:10:44.203-07:00@Steve, glad it worked. Also you can check out th...@Steve, glad it worked. Also you can check out the official Tungsten docs at www.continuent.com. Here's the URL for Vertica setup: <br /><br />https://docs.continuent.com/wiki/display/TEDOC/Replicating+Data+in+Real-Time+to+Vertica<br /><br />Feel free to log bugs if you see problems with the docs. <br /><br />p.s., Which Vertica version did you use?Robert Hodgeshttps://www.blogger.com/profile/05379726998057344092noreply@blogger.comtag:blogger.com,1999:blog-768233104244702633.post-87490094608610812132013-04-10T18:52:48.452-07:002013-04-10T18:52:48.452-07:00This article really got me moving, but I struggled...This article really got me moving, but I struggled to get fan-in replication with multiple mysql databases pointed at a single vertica instance. In case anyone else is facing this, the solution is to set up both masters as outlined in this article. Then link up your vertica database to the first master as above. To read from the second master, use <br /><br />tools/configure-service -C -a \<br /> --role=slave \<br /> --host=db2 \<br /> --master-thl-host=db3 \<br /> --local-service-name=myothersql2vertica \<br /> --service-type=remote \<br /> --release-directory=/opt/continuent/tungsten \<br /> --datasource-type=vertica \<br /> --datasource-user=dbadmin \<br /> --datasource-password=secret \<br /> --datasource-port=5433 \<br /> --batch-enabled=true \<br /> --batch-load-template=vertica \<br /> --vertica-dbname=bigdata \<br /> --buffer-size=25000 \<br /> --svc-start myothersql2vertica<br /><br />Assuming db3 is your other master. You have to explicitly re-describe your vertica instance here instead of just using -datasource. Also, --release-directory is not the --home-directory you used before. It has another /tungsten on the end of it.Anonymoushttps://www.blogger.com/profile/04495553806374188221noreply@blogger.comtag:blogger.com,1999:blog-768233104244702633.post-29419268853959318322012-08-22T12:23:34.170-07:002012-08-22T12:23:34.170-07:00Thank you Robert for the answers!
I have managed t...Thank you Robert for the answers!<br />I have managed to replicate mysql to vertica by following your tutorial !! and it worked well !! <br /> But i have question ?!<br />What is going to happen when you drop data from mysql ? Would id drop in Vertica as well ? as it seems this is not happening !!asdhttps://www.blogger.com/profile/15741952407258407414noreply@blogger.comtag:blogger.com,1999:blog-768233104244702633.post-13152030250517300892012-08-21T09:22:27.686-07:002012-08-21T09:22:27.686-07:00@asd, you are welcome. We use Oracle CDC, which i...@asd, you are welcome. We use Oracle CDC, which is in turn based on Streams, so it's real-time. Have a look at the following documentation article on Oracle to MySQL replication on the Continuent website: https://docs.continuent.com/wiki/display/TEDOC/Replicating+from+Oracle+to+MySQL<br /><br />It should give you most of what you need to know.Robert Hodgeshttps://www.blogger.com/profile/05379726998057344092noreply@blogger.comtag:blogger.com,1999:blog-768233104244702633.post-22701229979601134682012-08-21T09:02:51.039-07:002012-08-21T09:02:51.039-07:00Thnak you for the fast reply !
Is there a more pr... Thnak you for the fast reply ! <br />Is there a more precise link to this product page ?<br /> And if i might ask how is it done ? Log miner or how ... give me a clue :)"" thx asdhttps://www.blogger.com/profile/15741952407258407414noreply@blogger.comtag:blogger.com,1999:blog-768233104244702633.post-5440190185759665182012-08-21T08:04:41.035-07:002012-08-21T08:04:41.035-07:00@asd, yes there is. Continent is working on repli...@asd, yes there is. Continent is working on replication from Oracle to data warehouses including Vertica. Replication from Oracle is a commercial product but the prices are reasonable. Please check the Continuent website at www.continuent.com.Robert Hodgeshttps://www.blogger.com/profile/05379726998057344092noreply@blogger.comtag:blogger.com,1999:blog-768233104244702633.post-26731837897493366332012-08-21T07:24:07.246-07:002012-08-21T07:24:07.246-07:00Is there a way to this in Oracle ?
Oracle to Ver...Is there a way to this in Oracle ? <br /> Oracle to Vertica ?!asdhttps://www.blogger.com/profile/15741952407258407414noreply@blogger.comtag:blogger.com,1999:blog-768233104244702633.post-24768319038174134342012-08-21T07:17:54.309-07:002012-08-21T07:17:54.309-07:00Is there an Oracle -> Vertica replication using... Is there an Oracle -> Vertica replication using the Tungsten replicator ? <br /> Or is there anything that can do this !?? <br />THX Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-768233104244702633.post-15877156648317362042012-08-03T11:27:20.367-07:002012-08-03T11:27:20.367-07:00@asd Thanks. I'm glad you liked the article. ...@asd Thanks. I'm glad you liked the article. Speaking of which, check the first article in this series for more about updates. The batch load algorithm converts updates into insert+delete. We first remove existing rows that need to be deleted and then insert new rows. It gives you the same effect as applying in serial order but without the slowness you would otherwise get with Vertica UPDATE statements.Robert Hodgeshttps://www.blogger.com/profile/05379726998057344092noreply@blogger.comtag:blogger.com,1999:blog-768233104244702633.post-43016739713071666452012-08-03T11:20:47.053-07:002012-08-03T11:20:47.053-07:00Great POST !!
But how would the replicator will d...Great POST !! <br />But how would the replicator will deal with "UPDATES" in Vertica ?<br />Since Vertica does not deal well with updates ?asdhttps://www.blogger.com/profile/15741952407258407414noreply@blogger.comtag:blogger.com,1999:blog-768233104244702633.post-61937690554050566732012-08-03T11:19:17.067-07:002012-08-03T11:19:17.067-07:00Great article !!! Thank you for sharing !!
But ho...Great article !!! Thank you for sharing !! <br />But how would you deal with the "UPDATE" part in Vertica as we know Vertica does not deal well with this ???asdhttps://www.blogger.com/profile/15741952407258407414noreply@blogger.com