Richard Davies wrote: The UK has a good crop of technology pioneers in cloud computing - for example ElasticHosts, FlexiScale, Flexiant, OnApp - and also some strong government initiatives such as G-Cloud.
We will have to see whether this kind of technical leadership converts into swift mass-market adoption or not.
In many cases, the end of the year gives you time to step back and take stock of the last 12 months. This is when many of us take a hard look at what worked and what did not, complete performance reviews, and formulate plans for the coming year. For me, it is all of those things plus a time when I u...
Openadaptor is a software toolkit that may be classified as a lightweight Enterprise Application Integration (EAI) solution. It provides a configurable component framework for connecting various systems and middleware implementations. In less technical parlance, the components are akin to Lego building blocks that users can snap together to build adaptors, which themselves are the metaphorical glue or plumbing between applications. Originally written in 1997 to help a large financial organization integrate its applications with Message Oriented Middleware (MOM), while avoiding vendor lock-in in the process, Openadaptor has since joined the open source community and evolved to become one of the better recognized systems integration tools.
Through years of usage, however, several shortcomings have emerged. The initial architectural concepts of legacy Openadaptor predated the popularity of XML, and a canonical data representation called DataObjects became an integral part of the framework introducing certain performance overheads and overly strict rules as to what can, and what cannot, be exchanged between an adaptor's components. The experience of developing and supporting legacy Openadaptor has also highlighted several other shortcomings, such as the proliferation of custom code that led to the support hurdles, poorly defined and unclear customization and interface points, and the inevitable tendency toward bloat-ware. This was compounded by the fact that some of the inherent functionalities with time became superseded by industry standard equivalents (the clumsy, though effective, proprietary transaction manager would serve as an example).
Ismael,
Thanks for the feedback, it's always very useful to us. You should be able use stored procedures for updates - see the MapCallableStatementWriter (and corresponding map-stored-proc-writer.xml example, and their XML equivalents). In fact, if you can guarantee that the warehouse will always be based on Oracle it might be easiest to implement the 'upsert' funtionality in a stored procedure which the JDBC writer would invoke.
Having said that, with your comments in mind, we are looking at adding direct support for updates (and possibly 'upserts') with a view towards squeezing it into our upcoming 3.4.3 (November '08) release of openadaptor.
Also it's worth noting that the RawSQLWriter can achieve the same effect - it uses actual SQL strings which get directly executed against the db, albeit with a performance penalty (overhead of constructing the SQL manually, and not being able to compile it).
regards
Eddy Higgins, Kris Lachor (& the openadaptor dev team)
P.S. Note that www.openadaptor.org is the primary resource for openadaptor technical support. This affords the best visibility to the dev team, and other oa users.
#1
I Matos commented on 27 Oct 2008
Great article. Great tool. It seems a good fit to a new project I'm involved with.
The project will require extracting data from a ERP system to load into a data warehouse. The ERP system uses a DB2 UDB database, while the data warehouse will be based on an Oracle database. On a daily basis, it is necessary to extract all the new or updated records from selected tables on our ERP database to load on the data warehouse.
From the documentation and the samples, I found that it is very easy to INSERT data in a table, but could not find references to table UPDATES, or more specifically to the situation where an input record may be updating the corresponding record in the target table if it already exists, otherwise it will be inserted on the target table.
What would be the best approach to resolve this "UPSERT" type of problem using Openadaptor?
Eddy Higgins wrote: Ismael,
Thanks for the feedback, it's always very useful to us. You should be able use stored procedures for updates - see the MapCallableStatementWriter (and corresponding map-stored-proc-writer.xml example, and their XML equivalents). In fact, if you can guarantee that the warehouse will always be based on Oracle it might be easiest to implement the 'upsert' funtionality in a stored procedure which the JDBC writer would invoke.
Having said that, with your comments in mind, we are looking at adding direct support for updates (and possibly 'upserts') with a view towards squeezing it into our upcoming 3.4.3 (November '08) release of openadaptor.
Also it's worth noting that the RawSQLWriter can achieve the same effect - it uses actual SQL strings which get directly executed against the db, albeit with a performance penalty (overhead of constructing the SQL manually, and not being able...
I Matos wrote: Great article. Great tool. It seems a good fit to a new project I'm involved with.
The project will require extracting data from a ERP system to load into a data warehouse. The ERP system uses a DB2 UDB database, while the data warehouse will be based on an Oracle database. On a daily basis, it is necessary to extract all the new or updated records from selected tables on our ERP database to load on the data warehouse.
From the documentation and the samples, I found that it is very easy to INSERT data in a table, but could not find references to table UPDATES, or more specifically to the situation where an input record may be updating the corresponding record in the target table if it already exists, otherwise it will be inserted on the target table.
What would be the best approach to resolve this "UPSERT" type of problem using Openadaptor?
Many thanks for the help
I...
Yahoo’s critical negotiations with Alibaba to sell part of its stake in Alibaba back to the Chinese company have collapsed according to All Things Digital, a report later confirmed by CNBC.
Apparently the collapse includes Yahoo’s parallel and intertwined negotiations with Softbank t...
Can you bring services from the cloud to your customers faster and have them adopt it with ease of use or bring the power of bundled services to the fingertips of your clients without creating new rigid ‘apps stove pipes'? Do you want to prevent your business running away to public and...
The Internet highway may start looking like a proverbial New York traffic jam at rush hour soon.
Feel free to substitute any town you like because Cisco says there’s going to be a faster-than-expected 18x surge in worldwide mobile data traffic between 2011 and 2016.
That’s when mob...
OCZ Technology Group, a provider of high-performance solid-state drives (SSDs) for computing devices and systems, on Tuesday announced the Z-Drive R4 CloudServ PCI Express (PCIe) flash storage solution, designed to accelerate cloud computing applications and reduce operating expenses i...
Many organizations have embraced, or are considering, the benefits of cloud computing – speed, flexibility, increased expertise, shared workload, reduced costs, etc. The benefits are many – but so are the risks. What are the threats to cloud security? Which parties assume responsibilit...
SoftLayer Technologies on Tuesday announced the immediate worldwide availability of SoftLayer Object Storage, a redundant and highly scalable cloud storage service that allows users to easily store, search and retrieve data across the Internet, with optional CDN connectivity, or across...