What is NLP? In the words of an NLP Engineer

05 December 2017

Artificial Intelligence (AI) could be the most popular buzzword of 2017 but what does it actually refer to? Well many things, but that is far beyond the scope of our expertise so instead, we are on a mission to understand what experts in AI are actually working on and why they got into this field.

We start our series off with a focus on Natural Language Processing. This is a far-reaching field itself but can be broadly defined as the ability of a machine to analyse, understand and generate human language – but what does this mean in application? How can you kick-start your career in this field and what problems would you be tackling?

We caught up with Wei Liu to understand his fascination in the realm of NLP and what sort of projects he is working on as an NLP Engineer at Swiftkey to give you some insight.


Wei, what sparked your interest in the field of Natural Language Processing? What specific areas/techniques are particularly exciting for you?

 I stumbled upon the field of NLP after joining the NLP research team after my postgrad study. Before that I had thought NLP was something to do with linguistics and nothing to do with computer science or mathematics.

I personally find the recent word embedding techniques very exciting and promising. It offers a nice mathematical way of measuring relations between words.

We as humans instinctively realize that the words 'king' and 'queen' are similar, but it is very difficult for computers to learn that. Recent techniques such as 'word2vec' offers a way of measuring and comparing word similarities.


You’ve been at SwiftKey for 6+ years now, what sort of problems are you tackling there?

We deal with smart input. In essence; what can we do to help people to better type on smartphones and computers? There are many problems we are still tackling. For example, how can you support as many languages as possible? For smartphone devices, people often make typing mistakes due to the small screen and absence of a physical keyboard, how can we make smart corrections for them? Can we identify 'bad words'? The list goes on...


You keep up with the current research and news in this field – From your perspective, what do you think are key current issues in NLP research?

1. Language coverage. Most research is targeted at English. Part of the reason is that they have the most data available. Many other languages picked up the pace but there are many more are simply ignored. I think more international effort should be put to cover as many languages as possible, to make the NLP advancement available to all.

2. Language understanding. We have come a long way, but we are still far from developing true intelligent systems that understand language. For example, identifying sarcasm and humour.


In your opinion, what industries are due to be disrupted by NLP and why?

In my opinion it would be industries that are slow to adapt new technologies. For example, financial services, banks have lots of textual data/records that are underutilized, they are also resilient to latest technology due to many concerns.

Another area is health service sector, they require expensive expert knowledge. If some of the process can be automated with NLP, for example, identifying similar symptoms and diagnosis, we can greatly reduce the burden on health professionals and improve efficiency. I know IBM Watson team are doing interesting research in this area.


Finally, any advice to aspiring NLP Engineers?

Now is a very exciting and interesting time. Apart from the usual advice of keeping up with the state-of-the-art research and news, I'd also advice people to learn in a wider machine learning domain. Many NLP techniques are first introduced to solve image recognition and speech recognition problems.

If you are looking to hire or for a new career opportunity within Machine Learning or NLP then please contact hannah@campbell-north.com

View all blog posts