A Deep Choice Model for Hiring Outcome Prediction in Online Labor Markets


  • Yixuan Ma Beijing Jiaotong University
  • Zhenji Zhang Beijing Jiaotong University
  • Alexander Ihler University of California Irvine


deep choice model, hiring decision, convolutional neural network, conditional logit model, online labor markets


A key challenge faced by online labor market researchers and practitioners is to understand how employers make hiring decisions from many job bidders with distinct attributes. This study investigates employer hiring behavior in one of the largest online labor markets by building a datadriven hiring decision prediction model. With the limitation of traditional discrete choice model (conditional logit model), we develop a novel deep choice model to simulate the hiring behavior from 722,339 job posts. The deep choice model extends the classical conditional logit model by learning a non-linear utility function identically for each bidder within of the job posts via a pointwise convolutional neural network. This non-linear mapping can be straightforwardly optimized using stochastic gradient approach. We test the model on 12 categories of job posts in the dataset. Results show that our deep choice model outperforms the linear-utility conditional logit model in predicting hiring preferences. By analyzing the model using dimensionality reduction and sensitivity analysis, we highlight the nonlinear combination of bidders’ features in impacting employers’ hiring decisions.


Abhinav, K.; Dubey, A.; Jain, S.; Virdi, G.; Kass, A.; Mehta, M. (2017). Crowdadvisor: A framework for freelancer assessment in online marketplace, Proceedings of the 39th International Conference on Software Engineering: Software Engineering in Practice Track, Buenos Aires, Argentina, 93-102, 2017. https://doi.org/10.1109/ICSE-SEIP.2017.23

Agrawal, A.; Lacetera, N.; Lyons, E. (2016). Does standardized information in online markets disproportionately benefit job applicants from less developed countries? Journal of international Economics, 103, 1-12, 2016. https://doi.org/10.1016/j.jinteco.2016.08.003

Akiva, M. E.; Lerman, S. R.; Lerman, S. R. (1985). Discrete choice analysis: theory and application to travel demand, MIT press, vol. 9, 1985.

Boskin, M. J. (1974). A conditional logit model of occupational choice. Journal of Political Economy, 82(2, Part 1), 389-398, 1974. https://doi.org/10.1086/260198

Brynjolfsson, E.; Hitt, L. M. (2003). Computing productivity: Firm-level evidence. Review of economics and statistics, 85(4), 793-808, 2003. https://doi.org/10.1162/003465303772815736

Chan, J.; Wang, J. (2017). Hiring preferences in online labor markets: Evidence of a female hiring bias, Management Science, 64(7), 2973-2994, 2017. https://doi.org/10.1287/mnsc.2017.2756

Chollet, F. (2015). Keras. https://keras.io/, 2015.

Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, USA, 1251-1258,. 2017. https://doi.org/10.1109/CVPR.2017.195

Sousa, W.; Montevechi, J.; Miranda, R.; Rocha, F.; Vilela F. (2019). Economic Lot-size using machine learning, parallelism, metaheuristic and simulation, 18(2), 205-216, 2019. https://doi.org/10.2507/IJSIMM18(2)461

Fernández-Delgado, M.; Cernadas, E.; Barro, S.; Amorim, D. (2014). Do we need hundreds of classifiers to solve real world classification problems? The Journal of Machine Learning Research, 15(1), 3133-3181, 2014.

Goodfellow, I.; Bengio, Y.; Courville, A.; Bengio, Y. (2016). Deep learning, MIT press Cambridge, Vol.1, 2016.

Hagenauer, J. and Helbich, M. (2017). A comparative study of machine learning classifiers for modeling travel mode choice. Expert Systems with Applications, 78, 273-282, 2017. https://doi.org/10.1016/j.eswa.2017.01.057

Hensher, D. A.; Ton, T. T. (2000). A comparison of the predictive potential of artificial neural networks and nested logit models for commuter mode choice. Transportation Research Part E: Logistics and Transportation Review, 36(3), 155-172, 2000. https://doi.org/10.1016/S1366-5545(99)00030-7

Hoffman, S. D.; Duncan, G. J. (1988). Multinomial and conditional logit discrete-choice models in demography. Demography, 25(3), 415-427, 1988. https://doi.org/10.2307/2061541

Horton, J. J.; Tambe, P. (2015). Labor economists get their microscope: big data and labor market analysis. Big data, 3(3), 130-137, 2015. https://doi.org/10.1089/big.2015.0017

Huws, U.; Joyce, S. (2016). Crowd working survey: Size of the uk's 'gig economy' revealed for the first time. University of Hertfordshire, 2016.

Ji, S.; Xu, W.; Yang, M.; Yu, K. (2013). 3d convolutional neural networks for human action recognition. IEEE transactions on pattern analysis and machine intelligence, 35(1), 221-231, 2013. https://doi.org/10.1109/TPAMI.2012.59

Klein, G. A.; Orasanu, J. E.; Calderwood, R. E.; Zsambok, C. E. (1993). Decision making in action: Models and methods, Ablex Publishing, 1993.

Kokkodis, M.; Papadimitriou, P.; Ipeirotis, P. G. (2015). Hiring behavior models for online labor markets. Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, Shanghai, China, 223-232, 2015. https://doi.org/10.1145/2684822.2685299

Krizhevsky, A.; Sutskever, I.; Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, Nevada, USA, 1097- 1105, 2012.

Kuek, S. C.; Paradi-Guilford, C.; Fayomi, T.; Imaizumi, S.; Ipeirotis, P., Pina, P.; Singh, M. (2015). The global opportunity in online outsourcing, Technical report, World Bank, Washington, DC, 2015.

LeCun, Y.; Bengio, Y. (1995). Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks, 3361(10), 255-258, 1995.

LeCun, Y.; Bengio, Y.; Hinton, G. (2015). Deep learning, Nature, 521(7553), 436-444, 2015. https://doi.org/10.1038/nature14539

LeCun, Y.; Bottou, L.; Bengio, Y.; Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324, 1998. https://doi.org/10.1109/5.726791

Lazear, P.; Shaw, L.; Stanton, T. (2016). Who gets hired? the importance of finding an open slot, Technical report, National Bureau of Economic Research, 2016. https://doi.org/10.3386/w22202

Lhéritier, A.; Bocamazo, M.; Delahaye, T.; Acuna-Agost, R. (2019). Airline itinerary choice modeling using machine learning. Journal of Choice Modelling, 31, 198-209, 2019. https://doi.org/10.1016/j.jocm.2018.02.002

Lin, M.; Chen, Q.; Yan, S. (2013). Network in network. arXiv preprint arXiv:1312.4400, 2013.

Lin, W.-Y.; Hu, Y.-H.; Tsai, C.-F. (2012). Machine learning in financial crisis prediction: a survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 42(4), 421-436, 2012. https://doi.org/10.1109/TSMCC.2011.2170420

Ma, Y.; Zhang, Z.; Ihler, A.; Pan, B. (2018). Estimating warehouse rental price using machine learning techniques. International Journal of Computers Communications & Control, 13(2), 235- 250, 2018. https://doi.org/10.15837/ijccc.2018.2.3034

Maaten, L. V. D.; Hinton, G. (2008). Visualizing data using t-sne. Journal of machine learning research, 9, 2579-2605, 2008.

Mali, P.; Kuzmanovic, B.; Nikolic, M.; Mitic, S.; Terek, E. (2019). Model of leadership and entrepreneurial intentions among employed persons. International Journal of Simulation Modelling, 18(3), 385-396, 2019. https://doi.org/10.2507/IJSIMM18(3)471

McFadden, D. (1974). Conditional logit analysis of qualitative choice behavior. New York: Academic Press, Chapter 4, 1974.

McFadden, D. (2001). Economic choices. American economic review, 91(3), 351-378, 2001 https://doi.org/10.1257/aer.91.3.351

Moreno, A.; Terwiesch, C. (2014). Doing business with strangers: Reputation in online service marketplaces. Information Systems Research, 25(4), 865-886, 2014. https://doi.org/10.1287/isre.2014.0549

Mottini, A.; Acuna-Agost, R. (2017). Deep choice model using pointer networks for airline itinerary prediction. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada, 1575-1583, 2017. https://doi.org/10.1145/3097983.3098005

Nam, D.; Kim, H.; Cho, J.; Jayakrishnan, R. (2017). A model based on deep learning for predicting travel mode choice. Proceedings of the Transportation Research Board 96th Annual Meeting Transportation Research Board, Washington, DC, USA, 8-12, 2017.

Olugboyega O. (2016). Examining the protocols and platforms adopted for building information modelling processes by Nigerian construction professionals Journal of System and Management Sciences, 6(4), 1-45, 2016.

Omrani, H. (2015). Predicting travel mode of individuals by machine learning. Transportation Research Procedia, 10, 840-849, 2015. https://doi.org/10.1016/j.trpro.2015.09.037

Otsuka, M.; Osogami, T. (2016). A deep choice model. AAAI, Phoenix, Arizona USA, 850-856, 2016.

Pan, B.; Hsu, K.; AghaKouchak, A.; Sorooshian, S. (2019). Improving precipitation estimation using convolutional neural network. Water Resources Research, 55(3), 2301-2321, 2019. https://doi.org/10.1029/2018WR024090

Srivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1), 1929-1958, 2014.

Yoganarasimhan, H. (2013). The value of reputation in an online freelance marketplace. Marketing Science, 32(6), 860-891, 2013. https://doi.org/10.1287/mksc.2013.0809

Freelancer.com (2019). Freelancer.com reveals the fastest growing online jobs in 2018. http://www.freelancer.com. Accessed April 30 2019.

Upwork (2019). Upwork to report first quarter 2019 financial results on may 8, 2019. https://investors.upwork.com/news-releases. Accessed April 30 2019.



Most read articles by the same author(s)

Obs.: This plugin requires at least one statistics/report plugin to be enabled. If your statistics plugins provide more than one metric then please also select a main metric on the admin's site settings page and/or on the journal manager's settings pages.