Predicting a Job Title from a Job Description
The obvious first results to look at are what the model was trained to do: predicting a job title for a job description. This is a job description for an infrastructure engineer that was not part of the training set:
The job title is included in this job description and is in fact mentioned three times. There are also a lot of detailed duties and skills included for the position. Let’s create a second job description by replacing the job title “infrastructure engineer” with “person” and removing the two other references:
Now we run the job title prediction model on both job descriptions and compare the resulting embeddings with the learned job title embeddings from the model using the cosine similarity. These are the 10 most likely job titles under both scenarios:
|Original job description||Without “infrastructure engineer”|
|1||infrastructure engineer||linux engineer|
|2||linux engineer||devops engineer|
|3||systems engineer||linux systems engineer|
|4||windows engineer||systems engineer|
|5||devops engineer||systems administrator|
|7||linux infrastructure engineer||senior devops engineer|
|8||automation engineer devops||linux systems administrator|
|9||devops infrastructure engineer||devops consultant|
|10||linux systems engineer||build automation engineer|
For the original vacancy the best matching job title was “infrastructure engineer”, which happens to be the correct one. However, most other job titles in the top 10 make sense, although the “windows engineer” is a bit of an outlier.
For the modified job description, the best matching job title is “linux engineer”. It looks like the other 9 job titles would also be acceptable for this scenario. This example shows that the model was able to extract the job title, but it also came up with a meaningful prediction when we removed the actual job title from the job description. This suggests that the model was using duties, skills and other types of information in the description to come up with a prediction when the more direct information was missing.
This is particularly interesting because this was learnt largely from scratch as the model was fed with essentially raw words. All we did was tokenizing the job description into words, lowercasing and that’s it.
Another fact worth mentioning is that the vacancy above had the job title “engineering manager/ project engineer” associated with it. While the job description clearly mentions that the employer is looking for an “infrastructure engineer”, the correct job title was “engineering manager/ project engineer”. As with any real-world scenario, there is a certain amount of noise in the data. Despite this fact, we were quite satisfied with the results we obtained.
The practical applications of this are many, some of which will provide user assistance in platform soon. In simple terms however, it provides us with a mechanism to detect when a job title is probably not an ideal label for a particular vacancy and what a more suitable or common is.
Cleaning Job Titles or User Input
While the model was trained on job description data, we wanted to test it on data that is structured differently: simple queries and raw job titles. These cases have in common that they usually include some irrelevant words and often synonym mappings apply. Some of the examples we tried and resulting job title:
|Input||Predicted job title|
|Babysitter St Martins Primary School||babysitter|
|Part-time Library Assistant||library assistant|
|Administrative Assistant – Health Services||administrative assistant|
|Merchandiser (Personal Car)||merchandiser|
|Sr HR Administrator||human resources administrator|
|Call Center Representative I, Full Time, Oscoda, Michigan||call center representative|
|2016-17 Administrative Assistant III – Quail Run ES||administrative assistant|
|MAIL HANDLER ASSISTANT||mail clerk|
|we are looking for a business administrative apprentice and a||business administration apprentice|
These test data are quite different from what the model was trained on, but it looks like applying the model to these shorter queries results in predictions that are quite acceptable. We found that under these circumstances the cosine similarity that is being calculated for each reference job title has an important role to play. It provides a good measure of confidence for each prediction.
Removing the Obvious
Another type of job title prediction applies to situations where the actual job title is not directly mentioned in a short piece of input text. For humans it is usually clear what the job title should be, but can a model trained on a large number of job descriptions come up with a meaningful answer too?
This table shows some of the job titles that it came up with for various input:
|Input||Predicted job title|
|operative for groundworks||groundworker|
|babysitting St Martins Primary School||babysitter|
|we are Pemberton Truck Co and we are hiring regional drivers and a||truck driver|
|taking orders, extensive product knowledge, communication skills, dining skills||wait staff|
|our family is looking for someone to take care of our youngest child and a||nanny|
|we are urgently looking for a developer with a c++ background with a||c++ developer|
|Isuzu Technician (AK4679)||auto technician|
|we are looking for someone to manage a team of five software developers and a||software development manager|
|we are looking for someone to manage a team of five accountants and a||finance manager|
|we are looking for someone to manage a team of five sales representatives and a||sales manager|
These examples suggest that the model (to some extent) captures information about what someone with a particular job title usually does. For example, it predicts that someone who manages a team of accountants is probably a finance manager. By simply replacing “accountants” with “software developers” or “sales representatives” we jump to “software development manager” and “sales manager”, respectively. The model learned that a person who takes care of a child is often a nanny. When a truck company is looking for drivers, it is probably looking for truck drivers. Given the fact that “Isuzu technician” was not part of the job titles in our training data set, the prediction “auto technician” makes sense.
Finding Related Job Titles
A byproduct of the architecture of our model is that representations are obtained for all job titles in our training set. A job title is basically a short textual label for the responsibilities, duties, experience, etc… that come with a particular job. However, as the “infrastructure engineer” example showed, different people might assign different job titles to essentially the same job (or the same job title to different jobs).
While the CNN in our model generates a representation for a specific job description, we train the model to map those representations close to the representation of the corresponding job title. The representations in our model are embedding vectors that can easily be compared. That is how we did the predictions in the previous sections.
Since we use a large pool of job descriptions and learn to map those to the assigned job titles, we expect to obtain job title embeddings that provide us with a good idea of what a job usually represents.
In order to get a feeling for the quality of the job title embeddings we had a look at the closest neighbours for a particular job title in the embedding space. The table below shows the closest 7 job titles for a “statistician”, “humanties teacher” and “talent acquisition manager”:
|statistician||humanties teacher||talent acquisition manager|
|statistical analyst||humanities teacher||talent acquisition specialist|
|senior statistician||qualified humanities teacher||talent manager|
|biostatistician||religious studies teacher||talent acquisition partner|
|epidemiologist||qts nqt geography teacher||talent aquisition manager|
|research statistician||geography teacher secondary||talent acquisition consultant|
|statistical programmer||teacher of humanities||head of talent acquisition|
|data scientist||qualified history teacher||recruitment manager|
For “statistician” we find different variations as “statistical analyst”, “senior statistician” and “statistical programmer”. Also some related (sub)fields are detected: biostatistics and epidemiology. Some time ago there was quite a lot discussion going on in the statistical community (e.g. here and here) about the identity of statistics in data science and whether statistics is the same as data science. The appearance of “data scientist” in this list likely means that they share a significant amount of duties and skills in practice.
The spelling error in “humanties teacher” does not prevent the model from coming up with different variations and a list of different kinds of humanities teachers. Despite the fact that our approach does not take the actual characters in words into account, it seems that the model in combination with a large amount of data is tolerant to spelling errors (or at least to some extent). The same thing is observed for the “talent acquisition manager” example where we have a ”talent aquisition manager” in the nearest neighbours.
Some other examples show that relations between “AWS” – “cloud”, “solicitor” – “lawyer”, “property” – “real estate” are recognized:
|AWS architect||PA to general manager||property solicitor|
|aws consultant||personal assistant to general manager||property lawyer|
|aws solutions architect||pa to the general manager||senior property solicitor|
|aws developer||pa to senior management team||residential property solicitor|
|aws systems administrator||personal assistant to the general manager||property paralegal|
|aws devops||pa to senior management||conveyancing solicitor|
|devops architect||personal assistant office manager||commercial property solicitor|
|cloud architect||personal assistant pa||real estate solicitor|
While there are two job titles in “PA to general manager” we only see personal assistants and no general managers in the closest neighbours. As mentioned, we did not use the characters or words in the job titles, so this is likely because it was learned that those two professions have very different duties.
This article is part of a series on Using Deep Learning to Extract Knowledge from Job Descriptions. For more information, head to Using Deep Learning to Extract Knowledge from Job Descriptions.