Appen Named a Top 100 Company to Watch for Telecommuting Jobs in 2016

Appen is ranked #77 among the top 100 of companies hiring telecommuters in 2016

By |February 5th, 2016|Press|0 Comments

How to Write Great Rating Guidelines

6 Steps to High Quality Objective Data
By Ben Christensen & Carla Haces

Human evaluation is a key factor in improving web search, eCommerce search, and social media relevance algorithms. Humans are simply better than computers at managing subjectivity, understanding intent, and coping with ambiguity. But human evaluation comes with a risk: the very subjectivity we rely on humans for can also lead to inconsistent data. How to mitigate this? With great rating guidelines. Clearly-written guidelines can harness the subjective power of the human brain and transform it into objective data that can in turn be used to train computers to better understand that most important human of all—the end user.

A set of guidelines is a document containing instructions on using a given tool, or completing a given task. It can be very brief (a few pages) or as long as a book. This will depend on the complexity of the task, and the background of the evaluators. Guidelines are often written for non-technical (or semi-technical) individuals. It is very important that the wording used is clear, simple to understand, concise and overall effective in communicating the message without being overly technical, complex or redundant. Be brief but detailed.

By |February 4th, 2016|Press|0 Comments

Managing the Crowd: When to Use Managed Crowds vs. Crowdsourcing

By Ben Christensen
Director, Content Relevance

“When should I use crowdsourcing and when should I use a managed crowd?” This is the question anyone interested in staffing human annotation tasks should be asking, but many don’t because they don’t even know there are two different options. So let’s start there—defining the options. Assuming you need human annotation, for example for search relevance evaluation, there are two ways you can gather the necessary humans to do that work: 1. Crowdsourcing, where the task is made available to a large crowd without any training or management beyond a very limited set of task instructions and possibly a simple screening test; or 2. Managed crowds, where a smaller group is trained to complete the task accurately and overseen by a manager.

The power of crowdsourcing is in its numbers. You can accomplish a lot quickly because many hands make light work. A hundred thousand people can do quite a bit more than a hundred can. The cost is less because crowdsourcing typically pays only a few pennies per judgment. Most members of the crowd aren’t trying to make a living—they’re just trying to make a few extra bucks in their spare time. There’s usually little overhead involved in crowdsourcing because the crowd manages itself. You put the task out there, and if it’s interesting enough and pays enough, the crowd will get it done.

By |November 17th, 2015|Press|0 Comments

Appen’s Dorota Iskra to Speak at LT-Accelerate 2015

Appen is proud to announce that Dorota Iskra will be speaking at LT-Accelerate 2015 in Brussels, Belgium on Monday, November 23rd. Dorota’s presentation is entitled “Crowdsourcing in language data collection and annotation”.

Appen has been active in the area of data collection and annotation for over 15 years. Starting with a traditional approach of working locally, we have gradually moved to using web-based tools. This presentation focuses on advantages of web-based tools such as greater reach, lower cost, and enabling us to build an extensive network/crowd. On the surface crowdsourcing appears an attractive alternative for data collection and annotation, but poses a lot of challenges. In order to address these challenges, we have built our own crowd in a controlled and tested environment.

Dorota has extensive experience in speech and language technology where, after a period research, she has moved towards applications and language resources. She has worked in the telecom and software development industries and is currently responsible for the European business at Appen, a company collecting speech and language data and providing a wide range of linguistic services.


By |November 11th, 2015|Press|0 Comments

Appen to Exhibit at INTERSPEECH 2015 Conference in Dresden

Contact: Rachael Pappas, Appen

Language Technology Service Provider Appen Announces Expanded Offerings in Custom Linguistic Solutions

Sydney, Australia – September 2, 2015: Appen is pleased to announce that it will be exhibiting at the INTERSPEECH 2015 Conference in Dresden, Germany, September 6-10, in Booth 4.

Appen will be launching its expanded offerings in the area of on-demand linguist support provided by their Custom Linguistic Solutions (CLS) department.

CLS connects Appen’s clients with a premier network of native-speaker linguists and subject matter experts around the globe, for development, testing and customization of language technologies and applications. Appen’s solutions are tailored to fit their clients’ needs, whether they need on-site or remote support, on an ongoing basis or simply on demand.

For more information about Appen’s products and services, visit us at

About Appen: Appen is a global leader in language, search and social technology. With capability in more than 150 languages, we work with leading technology companies to expand into new global markets. Appen has offices in the US, Australia, and the Philippines, with remote employees all over the world. Appen was named in Forbes Magazine as one of the “Top 100 Companies Offering Flexible Jobs in 2015.”

By |September 4th, 2015|Press|0 Comments

Appen and Best Doctors Partnering in the IBM Watson Ecosystem

Contact: Rachael Pappas, Appen and Justin Joseph, Best Doctors

Appen and Best Doctors Partnering in the IBM Watson Ecosystem

Sydney, Australia and Boston, Mass – June 30, 2015: Appen, a global linguistic services firm, and Best Doctors, a global health services company that helps people get the right diagnosis and treatment plan, are proud to announce they have partnered through the IBM Watson Ecosystem to enhance the Best Doctors knowledge base of medical information.

Appen is a member of the Watson Ecosystem as a Talent Hub partner – a group of skilled professionals who provide specialized support to app-building partners to help them turn their creative visions into market-ready products and services. Specifically, Appen provides linguistic support for Watson application developers, including the development of language resources, such as question and answer pairs, lexical data and annotated corpora for training and tuning purpose.

“I am so pleased with our partnership with Appen,” said Cara Samia, DirectorProduct Management, Best Doctors. “Together, with IBM Watson, we are laying the crucial groundwork to enhance the Best Doctors knowledge base so that we may continue to improve and update our offering to Best Doctors members worldwide.”

“Years ago I started my career in IBM’s research division,” said Lisa Braden-Harder, Appen CEO. “Today, I’m excited to be a part of the future of technology, working together with IBM Watson and Best Doctors to integrate data, language and healthcare.”

Members of the Watson Ecosystem are bringing a variety of cognitive based apps to market to transform industries – from healthcare, to music, shopping and beyond. Developers and entrepreneurs building Watson apps have access to a variety of ecosystem resources such as the Watson Developer Cloud, and the Watson Talent Hub, among other resources.

About Appen
Appen is a […]

By |June 29th, 2015|Press|0 Comments

Appen’s Dorota Iskra to Speak at CLIN25 in Antwerp

Appen is proud to announce that Dorota Iskra will be speaking at the 25th Meeting of Computational Linguistics in the Netherlands (CLIN25) at the City Campus of the University of Antwerp on February 5-6, 2015.  Dorota’s presentation is entitled  “Approach to Non-standardised Languages in Asian and African Markets.”

As a global provider of language resources and linguistic services, Appen often has to work with languages where no established standard is present for orthography, vocabulary or grammar. This problem, although it may concern some minorities’ languages in Europe, becomes only apparent in its full dimensions when entering Asian and African markets. Examples are Arabic, Pashto, Urdu and various Indian languages.

This presentation describes the problems we have encountered when moving to Asian and African languages and proposes a methodology for establishing internal standards for non-standardised languages. In the initial phase of the project when creating orthographic transcription from speech we end up with multiple spellings for the same word. These are all added to the dictionary since there is no spell checker. Our linguistic expert checks the dictionary and makes decisions about which words should actually have the same spelling. As a result a rough spell checker is created which can be used in the next phase of the project. But because it is far from comprehensive, an exception list is kept. In multiple iterations the linguistic expert goes through the list and adds words to the dictionary.

Language technology expects consistency in the data which can only be achieved through a systematic approach in the face of lacking standards. The approach we propose has been developed and tested in a number of languages and with extensive volumes of data, not only by ourselves, but also by our clients who are […]

By |January 29th, 2015|Press|0 Comments

Appen’s Phil Hall to Speak at Mobile Voice Conference in San Jose

Appen is proud to announce that Phil Hall, SVP Language Resources, will be speaking at the Mobile Voice Conference in San Jose, California on April 20 and 21, 2015. Phil will be presenting on the topic of training data for speech recognition in a talk entitled, “The Value Versus Volume Continuum.”

Interested in learning more about how Appen works with speech recognition technology? Click here to read our case study on our work for the automotive industry.

By |January 29th, 2015|Press|0 Comments

Appen Named in Forbes Magazine as One of the Top 100 Companies Offering Telecommuting Jobs in 2015

Exciting news for Appen! For the second year in a row, Appen has been named in Forbes Magazine as one of the Top 100 Companies Offering Telecommuting Jobs. Forbes sourced its data from a list published by FlexJobs in January, 2015.

Following is an excerpt from the story: “The positive impact remote work options can have on a company’s bottom line and culture is increasingly being acknowledged by smart employers. The organizations on this list are among those leading the charge to show that remote work options are a part of successful 21st century workplaces,” says Sara Sutton Fell, founder and CEO of FlexJobs.

To view the story and the list on Forbes, please click here.

By |January 28th, 2015|Press|0 Comments

TAUS Review, January 2015 is Live!

The January 2015 issue of the TAUS Review, the online magazine published by TAUS, is live! Appen is a proud member of TAUS, a think tank and resource center for the global translation industry. To learn more about TAUS, click here, and to view the online magazine, click here.

By |January 19th, 2015|Press|0 Comments