TML for Rails & Globalize Gem

Suppose we have a Rails application with a large data set that we need to translate to a few languages. It is important that we store the data in our own database to preserve its integrity and provide internal searching capabilities. So how can we do this without having to invest too much time into building our own translation management platform?

Let’s use a simple Famous Quotes application as an example. This application contains 36,000 famous quotes that we would like to translate to a few languages. Our goal is to store all the quotes and their translations in our database and use crowdsourcing approach – asking users to help us translate the quotes or vote on existing translations. We will use Translation Exchange as a platform that will handle most of the work for us. All we have to do is handle the translation storage.

Using Globalize Gem

We need some kind of mechanism to store the translations in our database. Globalize gem is one of the best solutions for storing translations in Rails apps.

You can install the gem using the following url:
https://rubygems.org/gems/globalize3

For instructions on how to set it up, visit this url:
https://github.com/globalize/globalize

In short, simply add the gem to your Gemfile:

[ruby title=”Gemfile” class=”lang:ruby”]
gem ‘globalize’
[/ruby]

and run:

[shell]
$ bundle install
[/shell]

Configuring Our Application

In our application, all of the quotes data is stored in a single table, called “quotes”:

quotes

Name Type
id int
quote text
author string
authordata string

And we have a Quote model that loads and stores data in the database:

[ruby title=”app/models/quote.rb”]
class Quote quote_translations

Name Type
id int
quote_id int
locale string
quote text
author string
authordata string

To access the Quote data in other languages, we simply do:

[ruby]



[/ruby]

Similarly, to store the data in other locales, we use:

[ruby]
Globalize.with_locale(current_locale) do
quote.update_attributes(quote: params[:quote])
end
[/ruby]

Synchronizing Data with Translation Exchange

So now that we have a mechanism for storing and retrieving quotes data, let’s integrate our app with Translation Exchange and use its Translation Center and machine translations for getting the actual translations.

Besides translating the data, we may want to translate the chrome of the application (the static labels, buttons, links). We will use TML for Rails to do both chrome translation and data translation. So let’s first add the TML gem to our app.

Simply add the gem to your Gemfile:

[ruby title=”Gemfile” class=”lang:ruby”]
gem ‘tml-rails’
[/ruby]

and run:

[shell]
$ bundle install
[/shell]

Create a new initializer file with the following content:

[ruby title=”config/initializers/tml.rb” class=”lang:ruby”]
Tml.configure do |config|
config.application = {
key: YOUR_APPLICATION_KEY,
token: YOUR_APPLICATION_TOKEN
}

config.cache = {
enabled: true,
adapter: ‘rails’
}
end
[/ruby]

To learn more about all the available options of TML, visit:
https://translationexchange.com/docs/sdk/ruby

You can follow the above guide to see how to translate the chrome of your application. In this post, we will focus on how to synchronize the data with Translation Exchange. To do so, let’s create a new rake task with the following code:

[ruby title=”lib/tasks/quotes.rake” class=”lang:ruby”]
namespace :quotes do

desc ‘Exports quotes to Translation Exchange’
task :export => :environment do
Tml.session.init

buffer = []
buffer_size = 50

# if you don’t need to get the response data to store for reference, then realtime should be set to false
options = {
realtime: false,
dynamic: true
}

index = 0
Quote.find_each do |quote|
index += 1

# Quotes consist of 3 parts that need translations: quote, author, authordata
quote_attributes = {
source: {
name: “Quote #{quote.id}”,
id: “/quotes/#{quote.id}”
},
keys: [{
label: quote.author,
locale: ‘en’,
id: “/quotes/#{quote.id}/author”
}, {
label: quote.quote,
locale: ‘en’,
id: “/quotes/#{quote.id}/quote”
}]
}

# if author data is present submit it as well
unless quote.authordata.blank?
quote_attributes[:keys] 0
register_keys(buffer, options)
end
end

desc ‘Imports quotes translations from Translation Exchange’
task :import => :environment do
Tml.session.init

interval_start = Date.today – 30.days

params = {since: interval_start, locales: [‘ru’], source: ‘quotes’}

# paginate through results
data = fetch_translations(params)

imported = 0
while data[‘pagination’][‘current_page’] element[‘translation’][‘label’])
imported += 1
end
end
end

data = fetch_translations(params, data[‘pagination’][‘current_page’] + 1)
end

puts “Total imported #{imported} translations…”
end

private

def register_keys(buffer, options)
Tml.session.with_block_options(:live => true) do
Tml.session.application.api_client.post(‘sources/register_keys’, {
:source_keys => buffer.to_json,
:options => options.to_json
})
end
end

# Fetches translations from Translation Exchange
def fetch_translations(params, page = 1)
JSON.parse(
Tml.session.application.api_client.get(
‘sources/sync_translations’, params.merge(page: page), {raw: true}
)
)
end
end
[/ruby]

Now we can run the following rake task to sync up all the translation keys using:

[shell]
$ bundle exec rake quotes:export
[/shell]

And import translations using:

[shell]
$ bundle exec rake quotes:import
[/shell]

The following sections will provide more details on the methods used in the rake tasks above.

Registering Translation Keys

This method allows you to register the content of your application with Translation Exchange.

POST: https://api.translationexchange.com/v1/sources/register_keys

Parameters

Name Type Description
source_keys array Translation source/key objects
options hash Additional options: realtime (true/false)

source_keys

The source keys must be an array of the following objects:

[js]
{
source: SOURCE_OBJECT,
keys: [KEY_OBJECT,…]
}
[/js]

SOURCE_OBJECT contains the human readable source name and a source id to identify the source during import. For instance, in the above quotes application, we used:

[js]
{
name: “Quote #{quote.id}”,
id: “/quotes/#{quote.id}”
}
[/js]

KEY_OBJECT contains translation key information, such as original label, locale and a system id.

[js]
{
label: quote.author,
locale: ‘en’,
id: “/quotes/#{quote.id}/author”
}
[/js]

Sources allow us to group many keys together so it would be easier to organize and retrieve them later. In our examples, we wanted to create a source object for every quote, where quote’s text, author and authorinfo were all translation keys under the quote.

[js]
{
source: {
name: “Quote #{quote.id}”,
id: “/quotes/#{quote.id}”
},
keys: [{
label: quote.author,
locale: ‘en’,
id: “/quotes/#{quote.id}/author”
}, {
label: quote.quote,
locale: ‘en’,
id: “/quotes/#{quote.id}/quote”
}]
}
[/js]

options

If you don’t need to store the translation key reference data in your database, then you should always set the realtime option to false. This will significantly speed up the key export. If you do set the realtime to true, you will get the following information back:

[js]
{
:source => {
:id => “/quotes/#{quote.id}”,
:key => “dsklfsadfjaslkdjfalksjdflaksjdf”
},
:keys => [{
:id => “/quotes/#{quote.id}/author”,
:key => “sadlfkjasdljfkalskdjflaksjdflkajdfl”
}, {
:id => “/quotes/#{quote.id}/quote”,
:key => “sadlfkjasdljfkalskdjflaksjdflkajdfl”
}]
}
[/js]

If you store the reference key data, then you can access our public api to get individual keys translations:

GET: https://api.translationexchange.com/v1/translation_keys/#{KEY}/translations?locale=ru&access_token=…

A much faster and better approach is to synchronize/extract translations in bulk using the following method.

Syncing Translations

This method allows you to extract translations from Translation Exchange from your project’s translation memory.

GET: https://api.translationexchange.com/v1/sources/sync_translations

Parameters

Name Type Description
since date/time Format: 2015-12-01 10:30. Start date from which to get translations.
until date/time Format: 2015-12-01 10:30. End date until which to get translations.
locales array Comma separated list of locales. Only pull translations for the specified locales.
source string Only pulls translations that belong to translation keys matching a specific source key.
translators array Comma separated list of translator ids. Only pulls translations submitted by the specified translators.
imported boolean Format: true/false. Only pulls translations for imported translation keys.Keys that have been imported from files.
machine boolean Format: true/false. Only pulls translations that have been machine translated.
locked boolean Format: true/false. Only pulls translations that have been locked either by a reviewer or through purchasing from a 3rd party vendor.
purchases boolean Format: true/false. Only pulls translations that have been purchased from a 3rd party vendor.

The data from this call is returned in the following format:

[js]
{
“results”: [TRANSLATION_OBJECT],
“pagination”: PAGINATION_DATA
}
[/js]

Where TRANSLATION_OBJECT provides information about the original translation key and translation:

[js]
{
“id”: 3001,
“locked”: true,
“translation_key”: {
“id”:234210,
“key”:”881040e83278e40d7dba335ea73e6863″,
“label”:”Austrian psychiatrist & psychologist (1870 – 1937)”,
“locale”:”en”,
“word_count”:3,
“length”:51,
“created_at”:”2015-12-17 02:12:03″
},
“translation”: {
“id”:1329021,
“locale”:”ru”,
“label”:”Австрийский психиатр и психолог (1870 – 1937)”,
“rank”:1,
“machine”:true,
“created_at”:”2015-12-27 00:56:55″
},
“sources”:[“/quotes/16388/authordata”],
“created_at”:”2015-12-27 00:56:55″,
“updated_at”:”2015-12-27 00:56:55″
}
[/js]

Here is a full example of the data set:

[js]
{
“results”: [
{
“id”: 3001,
“translation_key”: {
“id”:234210,
“key”:”881040e83278e40d7dba335ea73e6863″,
“label”:”Austrian psychiatrist & psychologist (1870 – 1937)”,
“locale”:”en”,
“word_count”:3,
“length”:51,
“created_at”:”2015-12-17 02:12:03″
},
“translation”: {
“id”:1329021,
“locale”:”ru”,
“label”:”Австрийский психиатр и психолог (1870 – 1937)”,
“rank”:1,
“machine”:true,
“created_at”:”2015-12-27 00:56:55″
},
“locked”: true,
“sources”:[“/quotes/16388/authordata”],
“created_at”:”2015-12-27 00:56:55″,
“updated_at”:”2015-12-27 00:56:55″
}
],
“pagination”: {
“total_count”:100,
“current_page”:1,
“total_pages”:2,
“per_page”:50
}
}
[/js]

Notice that it is possible to have the same translation key be part of many sources, that’s why the “sources” attribute is an array of sources. It is completely up to you how you structure the sources – as long as you have a way to identify the data in your system and store the translations associations.

To view the source code for the above quotes application, please visit:

https://github.com/translationexchange/tml-rails-samples-quotes

If you have any questions about the above scripts, please contact our support: support@translationexchange.com

Get Started Today!

Create an account to get started now! No credit card required.

Get Started