Risk-Aware Adaptive Verification for Digital Onboarding: A Cost-Sensitive Contextual Bandit Framework with Calibrated Uncertainty

Tianjun Mo; Enmiao Feng; Gaike Wang; Pengfei Li

doi:10.54691/ezjatb36

Authors

Tianjun Mo Duke University, Durham, NC 27708, USA
Enmiao Feng Duke University, Durham, NC 27708, USA
Gaike Wang New York University, New York, NY 10012, USA
Pengfei Li Duke University, Durham, NC 27708, USA

DOI:

https://doi.org/10.54691/ezjatb36

Keywords:

Contextual bandits, cost-sensitive learning, conformal prediction, probability calibration, identity verification, account-opening fraud, off-policy evaluation, concept drift.

Abstract

Digital onboarding forces a bank to route every account-opening application through a ladder of identity-verification treatments whose operating costs, interdiction power and customer friction differ by orders of magnitude. The routing policy must be learned from bandit feedback-only the outcome of the treatment actually applied is ever observed-under extreme class imbalance, heavy-tailed monetary stakes, hard analyst-capacity limits and adversaries that adapt. We formulate risk-aware adaptive verification as a cost-sensitive contextual bandit with a structured reward decomposition and propose CARVE, which (i) turns a gradient-boosted risk score into an interval-valued calibrated probability using a bootstrap-aggregated Venn–ABERS predictor, (ii) pools treatment-effect regressions across the verification ladder so that the few hundred interdiction observations available at 1% fraud prevalence become usable, (iii) replaces the parametric LinUCB confidence radius with a block-aggregated conformal radius that measures epistemic rather than aleatoric uncertainty, and (iv) treats the two uncertainty sources asymmetrically-optimism over learnable treatment effects to drive exploration, calibrated pessimism over fraud risk to bound exposure-while a Lagrangian dual variable holds the manual-review rate at a prescribed capacity. On the NeurIPS-2022 Bank Account Fraud data, CARVE reduces expected cost per application by 19.6% relative to a tuned banded policy and by 9.7% relative to an offline cost-sensitive policy (p<0.01, 15 paired streams), closing to within 6.3% of a treatment-effect oracle, whereas unstructured LinUCB and Thompson sampling incur twice its cost. The conclusions transfer to card-not-present step-up authentication on a 285k-transaction real card log and hold across twelve perturbations of the response model.

Downloads

Download data is not yet available.

References

[1] Elkan, C. (2001). The foundations of cost-sensitive learning. In Proceedings of the 17th International Joint Conference on Artificial Intelligence (pp. 973–978).

[2] Sahin, Y., Bulkan, S., & Duman, E. (2013). A cost-sensitive decision tree approach for fraud detection. Expert Systems with Applications, 40(15), 5916–5923.

[3] Correa Bahnsen, A., Aouada, D., & Ottersten, B. (2015). Example-dependent cost-sensitive decision trees. Expert Systems with Applications, 42(19), 6609–6619.

[4] Correa Bahnsen, A., Stojanovic, A., Aouada, D., & Ottersten, B. (2014). Improving credit card fraud detection with calibrated probabilities. In Proceedings of the SIAM International Conference on Data Mining (pp. 677–685).

[5] Correa Bahnsen, A., Aouada, D., Stojanovic, A., & Ottersten, B. (2016). Feature engineering strategies for credit card fraud detection. Expert Systems with Applications, 51, 134–142.

[6] Ding, J., Shen, Z., & Liu, W. (2026). Game-theoretic cost-sensitive adversarial training for robust cloud intrusion detection against GAN-based evasion attacks. Applied Sciences, 16(8), Article 3944.

[7] Ping, W., Jiao, Y., Fan, H., & Zhang, X. (2026). Multimodal fraud detection in financial statements: A trimodal attention network with contrastive evidence chain construction. IEEE Access, 14, 80456–80468.

[8] Li, L., Chu, W., Langford, J., & Schapire, R. E. (2010). A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th International Conference on World Wide Web (pp. 661–670).

[9] Agrawal, S., & Goyal, N. (2013). Thompson sampling for contextual bandits with linear payoffs. In Proceedings of the 30th International Conference on Machine Learning (pp. 127–135).

[10] Chu, W., Li, L., Reyzin, L., & Schapire, R. E. (2011). Contextual bandits with linear payoff functions. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (pp. 208–214).

[11] Lattimore, T., & Szepesvári, C. (2020). Bandit algorithms. Cambridge University Press.

[12] Foster, D. J., & Rakhlin, A. (2020). Beyond UCB: Optimal and efficient contextual bandits with regression oracles. In Proceedings of the 37th International Conference on Machine Learning (pp. 3199–3210).

[13] Bietti, A., Agarwal, A., & Langford, J. (2021). A contextual bandit bake-off. Journal of Machine Learning Research, 22(133), 1–49.

[14] Bastani, H., Bayati, M., & Khosravi, K. (2021). Mostly exploration-free algorithms for contextual bandits. Management Science, 67(3), 1329–1349.

[15] Garivier, A., & Moulines, E. (2011). On upper-confidence bound policies for switching bandit problems. In Proceedings of the 22nd International Conference on Algorithmic Learning Theory (pp. 174–188).

[16] Dudík, M., Langford, J., & Li, L. (2011). Doubly robust policy evaluation and learning. In Proceedings of the 28th International Conference on Machine Learning (pp. 1097–1104).

[17] Swaminathan, A., & Joachims, T. (2015). Counterfactual risk minimization: Learning from logged bandit feedback. In Proceedings of the 32nd International Conference on Machine Learning (pp. 814–823).

[18] Agrawal, S., & Devanur, N. R. (2016). Linear contextual bandits with knapsacks. In Advances in Neural Information Processing Systems (Vol. 29, pp. 3450–3458).

[19] Platt, J. C. (1999). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In A. J. Smola, P. Bartlett, B. Schölkopf, & D. Schuurmans (Eds.), Advances in large margin classifiers (pp. 61–74). MIT Press.

[20] Zadrozny, B., & Elkan, C. (2002). Transforming classifier scores into accurate multiclass probability estimates. In Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 694–699).

[21] Kull, M., Silva Filho, T. M., & Flach, P. (2017). Beta calibration: A well-founded and easily implemented improvement on logistic calibration for binary classifiers. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (pp. 623–631).

[22] Niculescu-Mizil, A., & Caruana, R. (2005). Predicting good probabilities with supervised learning. In Proceedings of the 22nd International Conference on Machine Learning (pp. 625–632).

[23] Vovk, V., & Petej, I. (2014). Venn–Abers predictors. In Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence (pp. 829–838).

[24] Vovk, V., Petej, I., & Fedorova, V. (2015). Large-scale probabilistic predictors with and without guarantees of validity. In Advances in Neural Information Processing Systems (Vol. 28, pp. 892–900).

[25] Vovk, V., Gammerman, A., & Shafer, G. (2005). Algorithmic learning in a random world. Springer.

[26] Angelopoulos, A. N., & Bates, S. (2023). Conformal prediction: A gentle introduction. Foundations and Trends in Machine Learning, 16(4), 494–591.

[27] Angelopoulos, A. N., Bates, S., Fisch, A., Lei, L., & Schuster, T. (2024). Conformal risk control. In Proceedings of the 12th International Conference on Learning Representations.

[28] Chen, Z., Wang, M., Zeng, Z., & Ping, W. (2026). Uncertainty-aware financial forecasting: Leveraging conformal prediction for risk-adjusted models. IEEE Access, 14, 90053–90077.

[29] Saerens, M., Latinne, P., & Decaestecker, C. (2002). Adjusting the outputs of a classifier to new a priori probabilities: A simple procedure. Neural Computation, 14(1), 21–41.

[30] Hüllermeier, E., & Waegeman, W. (2021). Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine Learning, 110(3), 457–506.

[31] Madras, D., Pitassi, T., & Zemel, R. (2018). Predict responsibly: Improving fairness and accuracy by learning to defer. In Advances in Neural Information Processing Systems (Vol. 31, pp. 6147–6157).

[32] Mozannar, H., & Sontag, D. (2020). Consistent estimators for learning to defer to an expert. In Proceedings of the 37th International Conference on Machine Learning (pp. 7076–7087).

[33] Alves, J., Jesus, S., Sousa, L., Saleiro, P., & Bizarro, P. (2023). FiFAR: A fraud detection dataset for learning to defer. arXiv. https://arxiv.org/abs/2312.13218

[34] Sani, A., Lazaric, A., & Munos, R. (2012). Risk-aversion in multi-armed bandits. In Advances in Neural Information Processing Systems (Vol. 25, pp. 3275–3283).

[35] Rockafellar, R. T., & Uryasev, S. (2000). Optimization of conditional value-at-risk. Journal of Risk, 2(3), 21–41.

[36] Jin, Y., Yang, Z., & Wang, Z. (2021). Is pessimism provably efficient for offline RL? In Proceedings of the 38th International Conference on Machine Learning (pp. 5084–5096).

[37] Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014). A survey on concept drift adaptation. ACM Computing Surveys, 46(4), Article 44.

[38] Fan, H., Jiao, Y., Wang, M., Chen, J., & Yang, S. (2026). Fraud learns too: Continual graph learning under strategic adversarial drift in dynamic networks. Scientific Reports. https://doi.org/10.1038/s41598-026-60997-7

[39] Jesus, S., Pombal, J., Alves, D., Cruz, A. F., Saleiro, P., Ribeiro, R. P., Gama, J., & Bizarro, P. (2022). Turning the tables: Biased, imbalanced, dynamic tabular datasets for ML evaluation. In Advances in Neural Information Processing Systems (Vol. 35), Datasets and Benchmarks Track.

[40] Dal Pozzolo, A., Caelen, O., Johnson, R. A., & Bontempi, G. (2015). Calibrating probability with undersampling for unbalanced classification. In Proceedings of the IEEE Symposium Series on Computational Intelligence (pp. 159–166).

[41] Dal Pozzolo, A., Boracchi, G., Caelen, O., Alippi, C., & Bontempi, G. (2018). Credit card fraud detection: A realistic modeling and a novel learning strategy. IEEE Transactions on Neural Networks and Learning Systems, 29(8), 3784–3797.

[42] Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., & Liu, T.-Y. (2017). LightGBM: A highly efficient gradient boosting decision tree. In Advances in Neural Information Processing Systems (Vol. 30, pp. 3146–3154).

[43] Hardt, M., Price, E., & Srebro, N. (2016). Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems (Vol. 29, pp. 3315–3323).

[44] Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017). Inherent trade-offs in the fair determination of risk scores. In Proceedings of the 8th Innovations in Theoretical Computer Science Conference (pp. 43:1–43:23).