Scoring Machine system. Creating a scoring model
Creating a scoring model
The block "Creating a model" contains sections:
- Creation of a new model;
- List of models;
- Settings for creating a model;
- Advanced settings for creating a model.
In the "Settings for creating a model" section, you can configure the parameters for building a scoring model.
You can specify how exactly in the file that will be loaded to create the scoring model and test, “Good” and “Bad” results will be marked in the first column of the file. Or you can leave the standard names for good and bad, but then this is how they should be named in the files.
You can also select the quality level of the model. The higher the level, the better the model will be! But! If your list of data has weak dependencies, you need to select the appropriate model quality level. I recommend starting with level 2, which is the default. Tier 2 and Tier 3 are considered the best for most cases and lists. But you can try to build different models, with different settings, and after testing, choose the best option for yourself.
The quality level affects how influential attributes the system will look for when creating a model, which means that the higher the quality level of the model is, the more significant attributes will be searched for. But, if such attributes are not found or there are few of them, then the model will not be of high quality. Therefore, it is important to match the quality level of the model to the quality of the list of data itself. Again, I recommend starting at level 2 or 3.
Usually, to create a high-quality model, it is enough to find from 8 to 12 attributes. If in the end there are more than 20 attributes, then you can safely try to rebuild the model, but with a higher model quality in the settings.
To change the settings or restore the default, you can use the "Actions" button in the upper right corner.
In the "Advanced settings for creating a model" section, there is also the possibility of advanced settings for users with a Level 5 subscription, where it is possible to fine-tune the algorithm for selecting influential attributes for creating a model.You can read more about advanced settings here.
In the "Creating a new model" section, the process of creating a scoring model takes place.
It is important to note that each subscription has its own limits on the number of simultaneously saved models. Therefore, if this limit is exhausted at the beginning of building a new model, the system will independently delete the model and all tests for it, which was created earlier than the others. Therefore, in order to avoid such automatic deletion of the model, it is better to control the number of saved models yourself within the limit of the current subscription or change the subscription. You can control by removing unnecessary models yourself. When deleting a model, all tests for it (if any) will also be deleted.
To create a model, you need to select a file with data for analysis with the extension .xls or .xlsx on your computer and upload it to the Scoring Machine system. Then click on "Build New Model". The system will make a series of checks on the file, if any inconsistencies in the requirements for the file are found during the first observation, an error will be displayed. After checking the file for a short time, the Scoring Machine will start analyzing the file and display a notification.
The duration of model creation depends on the amount of data to be analyzed. The larger the data file, the longer the system will analyze and build the model. This means that if the same file is analyzed several times already, but with different settings for creating a scoring model, then you can safely delete unnecessary columns, for example, those that, after the first analyzes, showed little information value for the model.
A very important process before building a model is to prepare the file correctly so that the system can analyze it better.All file requirements can be found here.
After the Scoring Machine finishes creating a model, it will immediately be displayed in the general list of models, and an email will be sent to the user's email informing them that the model creation process has been completed. If the model is not created for some reason (for example, errors in the file, the presence of empty cells where they should not be), then an e-mail will also be sent to the user about this.
The Models List section displays already built and saved models in the Scoring Machine. The results are displayed on 10 models per page, to go to the next / previous / last / first page, you must click on the corresponding symbol at the bottom of the table.
To search for a specific model, if there are many, you can use the search. To do this, click on the "Search" button in the upper right corner.
To go to the desired model, just click on the line in the table with this model.
When switching to the model, the interface is divided into three sections:
- General information;- Scoring model;- Tests of the scoring model.
In the "General Information" section, you can find the name of the model, its description, when the model was created and when the last edit was made. It also displays the status of the model.
The name of the model and its description are specified by the user at will. These values are needed exclusively by the user in order to navigate what kind of model it is at one stage or another. You can change the model name and / or description with the "Actions". It is recommended to change them immediately after creating the model.
Also below on the page in the table all the attributes from the file are displayed, as well as their informational value. The higher the % in the information value, the more valuable and important the attribute. Accordingly, if the score is very low, then the importance of this attribute is insignificant, it is better not to use such attributes in the final model.
In the section "Scoring model" there will be an already formed model. Namely, all the attributes with the necessary data for each of them. To open an attribute with information on it, you must click on the line with the attribute.
The most important thing here is the value of the attribute and next to it is the total number of points, if this value is in the attribute when building a forecast. All other attribute parameters are informative primarily for advanced users who are deeply familiar with the construction of scoring for their own analysis, if necessary and desired.
Also, if the user has a desire, he can change the score for a specific attribute value, for this you need to click on the appropriate button next to the score and the changes will be saved in the model.This feature is designed to allow small adjustments that the user can know for sure. After your adjustments, you can also retest the scoring model.
More details about the values of all displayed parameters in each attribute are described here.
The "Scoring Model Tests" section groups tests by the open scoring model and displays the main information on the test results. Also, after opening the test directly from the model, you can go to the test page with the button.
With the "Actions" you can activate / deactivate the model to test it, edit the name and description, go to create a new test for the model, export all model data to an excel file, delete the model.