Insuretech: Fire Your Underwriter! Let's Build an Underwriting Engine

This article will take the concept of writing an underwriting engine allowing APIs and platforms to collect user information and be able to provide an accurate quote based on those values. As this application matures, I will introduce the ratings engine to a module that will analyze the collected data and make decisions related to underwriting and rates that would otherwise be made by a human underwriter. In this first article, we will write the basic structure of the engine and define its data points.

Source: Paramount Pictures

SPOILER ALERT: This is laying the foundation of an underwriting rules engine. Once we build out the engine, we can feed in any number of "applications" as policies. As the applications are processed, we will want to statistically analyze the quotes to look for any outliers and to validate that the data is normal. Once that is understood, we will want to start introducing claim payouts to offset our collected premiums and see how we can add logic that will allow the engine to make its own pricing decisions. I have a feeling this is going to be a bit of a journey. Please grant me some patience while I figure some of this stuff out :)

Insurance underwriters are professionals who evaluate and analyze the risks involved in insuring people and assets. Insurance underwriters establish pricing for accepted insurable risks. The term underwriting means receiving remuneration for the willingness to pay a potential risk. - Investopedia

Let's take this apart and apply the concept to software. By the end of this exercise, we should have a rough idea of the role of the underwriter and the software that supports the underwriter in making decisions. Ideally, we will be able to build a decision tree based on the questions and responses that the underwriter evaluates for an insurance applicant to help make decisions in the form of risk scoring.

I don't think it is necessary to get into the semantics of insurance policies, carriers, risks, underwriting, etc. Instead, I want to take a very simply concept and build out from there. Let's say that a carrier has developed a product in which they agree to insure property damage sustained for widgets. A person owning a widget wishes to purchase insurance for said widgets so that in the event of using the widget, it sustains accidental damage, and the widget owner would like the widget repaired or replaced. Simple enough.

For some reason, I always pictured a widget to look like the original Everlasting Gobstopper from the first Willy Wonka movie. Anyway, widgets usually sell for around $5,000, and owners who don't have that kind of cash, would like to secure their $5,000 investment by purchasing an insurance policy to cover accidental damage. Instead of building out a set of attributes, let's use this step to get into some code.

First, we'll need a database. This will be used to store widget attributes, the rating rules associated with those attributes, and policy information. Instead of building out a full database, we'll just use a JSON document, which should give us enough flexibility with the attributes to illustrate the concept.

Second, our code will read from the database, consume the information, and spit out a risk score in the form of a premium quote. We can get fancy with risk scores in the next article, but for now, let's KISS (keep it stupid simple). The other thing we want to build into this is the idea of extendability, as the data size increases for our widgets, we need the engine to scale automatically, instead of having to go back and re-write the code to accommodate it.

Last, we'll need an object that will represent an "applicant" widget. When we feed the application into the widget, it should process the application and return with a risk score, that we are considering here as a quote.

It will look something like this:

Application --> Engine --> Quote

Source: https://etc.usf.edu/clipart/41800/41849/function_41849.htm

Let's dive in!

The first step is to define the widget. We know that there are two attributes in this first example: purchase date and purchase price. Later, we'll introduce other concepts, but these are good for now. Our JSON will look like this:

{
    "widget_attributes": [{
        "purchase_date": "Date of Purchase",
        "purchase_price": "Purchase Price"
    }]
}

Basically, we want our code to read the database and pull the list of attributes in the widget_attributes table. Once the list is read in, we can iterate through the list and process each individually. The first method will read the database (JSON file) into memory and the second method will pull out the widget attributes.

import json

def read_json_file():
    filename = "u1.json"
    with open(filename, "r") as u_data:
        data = json.load(u_data)
        json_str = json.dumps(data)
        resp = json.loads(json_str)
    return resp


def get_widget_attributes(data):
    attrs = []
    for attributes in data['widgetdata']:
        for attribute in attributes:
            attrs.append(attribute)
    return attrs


def main():
    data = read_json_file()
    attrs = get_widget_attributes(data)
    print(rule)

if __name__ == '__main__':
    main()

The first part is done. We now have a way to read in attributes about our widget from our data source. As we expand the widget, we just have to add new attributes to the JSON.

The next step is to add a rule and the method to read the rule. We can expand our underwriting engine by adding rules to the JSON. I want the code to iterate through the list of attributes that was returned in the first method, then, using the first attribute, look up the corresponding rule in the rules engine that belongs to that attribute. For example, the first widget attribute is "purchase_price". I would like a rule named "purchase_price" defined in the rules engine. If we pass the purchase price to the rules engine, then it will process the rule accordingly. First, we need to write a method that will retrieve the rule based on the attribute passed, which should look something like the following:

def get_rule_for_attribute(data, attribute):
    ruleset = []
    for rules in data['enginerules']:
        try:
            for rule in rules[attribute]:
                ruleset.append(rule)
        except KeyError:
            pass
    return ruleset

def main():
    data = read_json_file()
    attrs = get_widget_attributes(data)
    for attr in attrs:
        rule = get_rule_for_attribute(data, attr)
    print(rule)

The main method has been modified to iterate through the attributes and call the function that will retrieve the corresponding rule for that attribute from the rules engine.

In my JSON, I am going to give it a very simple rule. I will define a parameter called "divisor", where we will use that value to divide anything passed to it, and another parameter called "rate" which will be some value that corresponds to that function. In other words, this rule will be:

quote = purchase_price / divisor * rate

We'll be more complicated rules in a bit, but for now, we are building a foundation, so best to start a bit simple.

The updated JSON:

{
    "widgetdata": [{
        "purchase_date": "Date of Purchase",
        "purchase_price": "Purchase Price"
    }],
    "enginerules": [{
        "purchase_price": [{
            "divisor": 100,
            "rate": 25.00
        }]
    }]
}

Underwriting 1: First Rule

We can write a test case to validate our rules as follows:

import unittest
import u1

class TestUnderwriting1(unittest.TestCase):

def test_get_rules(self):
    data = u1.read_json_file() # read the file
    attrs = u1.get_widget_attributes(data)
    for attr in attrs:
        rule = u1.get_rule_for_attribute(data, attr)
        for each in rule:
            self.assertEqual(each['divisor'], 100)

if name == 'main': unittest.main()

Underwriting 1: First Test Case

With the test case in place, we can add new rules and modify the code and continue to run tests to be sure we didn't break anything.

We now have a program that will retrieve a list of attributes and then retrieve the rule(s) associated with each attribute. It is now time to build the "engine", which is really just a function that consumes the rules and input variables to return a value. To kick it off, we need a function that will accept the rule and the value corresponding to that rule. In looking at what we have done so far, we would need to pass the rule for purchase_price and the corresponding value for purchase price. The "engine" will perform the math and return a value, which, in this case, is the quote.

Here's the updated JSON to add an "application". The application has one parameter: "purchase_price".

        ,
    "application": [{
        "purchase_price": 500.00
    }]

We now want to modify the code to read through the application and for each value in the application, look in the rules engine for the corresponding rule. Imagine that we have a front-end that will validate the application and make sure all of the required values are populated before we post this to the underwriting engine. I have added two new functions and modified the main method to process the application:

def run_rule_engine_for_value(rule, inval):
    quote = 0
    keys = rule.keys()
    for key in keys:
        if key == 'divisor':
            quote = inval/float(rule[key])*rule['rate']
    return quote

def get_quote_for_value_and_rules(rules, inval):
    quote = 0
    for rule in rules:
        quote = run_rule_engine_for_value(rule, inval)
    return quote


def main():
    data = read_json_file("u1.json")
    attrs = get_widget_attributes(data)
    application = get_values_from_application(data)
    for each in application:
        for key in each.keys():
            value = each[key]
            if key in attrs:
                rules = get_rules_for_attribute(data, key)
                quote = get_quote_for_value_and_rules(rules, value)
                print(quote)

Underwriting 1: First Rule

And, a new test case to assert whether the result matches the expectation:

    def test_get_quote(self):
        data = u1.read_json_file("u1.json")
        attrs = u1.get_widget_attributes(data)
        application = u1.get_values_from_application(data)
        for each in application:
            for key in each.keys():
                value = each[key]
                if key in attrs:
                    rules = u1.get_rules_for_attribute(data, key)
                    quote = u1.get_quote_for_value_and_rules(rules, value)
                    self.assertEqual(quote, 125.0)

Underwriting 1: First Test Case

OK, so the basic foundation is there. Here's the rub: Whatever math logic we are going to use to process the application, we have to build into the rules engine. I have added a simple value called "divisor", but we can add another pricing algorithm, which is base and base rate. In this step, there is one rate (base_rate) charged for any amount up to the "base", and then the normal base kicks in after that.

For example, let's say that for purchase_price, we are going to charge one rate for the first $100, and $50 thereafter. To do this, we need to add the pricing rule to the underwriting engine JSON repository:

    "enginerules": [{
        "purchase_price": [{
            "divisor": 100,
            "rate": 25.00,
            "base": 100.00,
            "base_rate": 50.00
        }]
    }],

And the method will be changed as follows:

def run_rule_engine_for_value(rule, inval):
    quote = 0
    keys = rule.keys()

    if 'base' in keys:
        inval = inval - rule['base']
        quote += rule['base_rate']
    if 'divisor' in keys:
        quote += inval/float(rule['divisor'])*rule['rate']

    return quote

The first step is going to decrease the purchase price by removing the base amount and charging a required $50. The second step will be to divide the inval, which should now be 400 by the divisor of 100, to give 4. Then we multiply that by the rate (4*25) and add that to the running total, which should now be $150.

And, a quick print gives the expected value:

Underwriting 1: First Rule

I think this is a good place to stop for this article. We now have the foundation for the underwriting rules engine written. We can add more rules and code more math logic, and I think this is a good blueprint on how to accomplish that, but for now, this should serve the purpose. In the next article, I will write a loader function to load in thousands of "applications" and the program will write the quotes for each application. Then, we'll take a look at how we can use Python to analyze those results.

Let me know your thoughts on Twitter.