Fintech Today
  • About
  • Banking
  • Insurance
  • Resources
    • COVID-19
Subscribe
No Result
View All Result
  • Digital Transformation
  • Customer Experience
  • Cybersecurity & Risk
  • Regulation & Compliance
  • Claims Management
Fintech Today
  • Digital Transformation
  • Customer Experience
  • Cybersecurity & Risk
  • Regulation & Compliance
  • Claims Management
No Result
View All Result
Fintech Today
No Result
View All Result
Home Banking

Edit and Validation the Sharp End of Data Governance

by Phil Teplitzky
February 27, 2018
in Banking
Reading Time: 6 mins read
A A
Data Governance
Share on FacebookShare on Twitter

There has been a great deal of interest of late in data governance and in the identification and definition of data elements of material business importance and in the stewardship of those elements so as to define the meaning, structure, and format of the data elements.

We have gone so far as to install data governance software to memorialize our definitions and provide applications and data structure designers with their content to be used in the development of edit and validation routines.  What we have often overlooked is creating an approach and methodology for defining the order in which the rules should be applied.

Edit and validation rules are the rules we instantiate in the code base to ensure that the data that we are using to create and update shared data structures are accurate, complete, authorized, timely and support our needs for being an ongoing concern (e.g., the well-defined, accepted and understood control objectives for data).  The problem has become more complicated as multiple applications update the same data structure(s).  The edit and validation code base must instantiate the edit and validation rules, the rules defined by the data governance function, in the same or at least analogous. It must be noted that each of the specific languages used to build the applications may have different syntaxes and hence require different logic structures to accomplish the same tasks.

To this end, we have created a taxonomy of edit and validation rules which defines the order in which they should be applied, an order that represents the greatest intrinsic value.  The taxonomy will aid in our understanding of how edit and validation are instantiated in an operational/transactional applications:

  1. Syntactical – the data elements is evaluated regarding its data type, structure, and adherence to the standard established and documented in the data governance repository:
    • Standard Data Types – in Third and some Fourth generation compiled languages Strong Data Typing is used (this is done so the Compiler can establish the Symbol Table, allocate storage and most important know how to execute logic and arithmetic calculations). Examples of traditional data types are:
      1. CHAR – Character, with the additional need to define the size, e.g., CHAR 50, Fifty characters long; fixed, blank filled left justified, it could also be VAR – Variable and Right justified; the programmer must define the syntactical specifications
      2. Numeric – size, and some decimals, e.g., NUMERIC 20.5 which would be 20 positions in length with 5 of them being to the right of the decimal point.
  • Vectors, Arrays, and other non-scaler structures
  • Abstract Data Typing – a new concept that allows the programmer to define new types of data structures and operations; for example, to create as n-dimensional Matrix of images or sounds. Other types of abstract data typing are unstructured data (e.g., movies, documents, sound).  These unstructured data types require edit and validation rules, methods and techniques that are very different than structured data.
  1. Semantical –
    • Intra-transactional Semantical – Semantic Editing as compared to Syntactical or Data Type editing uses the value or meaning of the data that has been input, the Occurrence Data. A few examples may make this more understandable:
      1. Zip Code Plus 4 provides, via a look up to a file provided by the Post Office, the correct address, including accepted standard abbreviations (e.g., RD for Road, PL for Place)
      2. Code Look Ups for company specific abbreviations and codes, e.g., SKU, catalog numbers and acceptable dimensions for shipping
  • Codes published by governmental agencies and standards groups
  1. Economic order and ship quantities and related standard references
  • Inter- or Historical Inference – in this case, the Semantic correctness of the input is evaluated based on historical behaviors. For example; if the customer has ordered the same SKU 20 time in the last 6 months and has consistently ordered a dozen of that SKU, and the order now comes in for 10 dozen (dozen being the order qauntity established by 2 above) the edit and validation algorithm, based on looking up prior behavior can make the inference that the customer wanted 1 dozen and asked for confirmation at the time of data entry.  This inference editing has become a standard differentiator for several internet providers; they remember your purchases and make recommendations or check the semantic of your order against their historical records.  It is also the basis for fraud detection in online transactions and credit cards, you cannot purchase in Tokyo and five minutes later make one in New York, the historical and geographical constraints do not work.

In summary, data governance function provides the rules needed to determine what the correct and acceptable definition, at the syntactic and semantic level of a data element, edit and validation code is how the rules get instantiated in the operational ecosystem.  Inconsistencies in instantiation result in the degradation of the accuracy, completeness, authorization, timeliness, and ability to support an ongoing concern requirement.  Or, to put it more colloquially, if the rules are not consistently applied across the ecosystem and in agreement with the data governance definitions the data is WRONG, and the business is at risk.

To resolve these conflicts organizations have established Methods, Standards, Design Techniques, Procedures and implemented Architectural Paradigms to enforce data governance definitions and result in edit and validation code that is consistent across multiple applications and data structures.

Like what Philip Teplitzky has to say? He has a monthly column on Fintech Today, which you can subscribe to here.

Tags: credit card fraudData ElementsData Governancedata governance softwareData ValidationEdit and Validation Rulesfraud detection

RELATED POSTS

Monetizing Data
Banking

Monetizing Data: Follow the Yellow Brick Road

April 4, 2018
Paradigm Shift
Banking

Paradigm Shift in Computing for the Financial Services Sector

January 30, 2018

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

TRENDING NOW

  • Cybercrime and Cyber Risk: Strategies for Protecting Your Organization

    503 shares
    Share 201 Tweet 126
  • What 2023 Means for Commercial Construction Lending

    515 shares
    Share 206 Tweet 129
  • Arizona’s Motor Vehicles Department (MVD) Makes Payment Offerings More Accessible

    657 shares
    Share 263 Tweet 164
  • AI in Financial Services: Where Does Ethics Fit In?

    553 shares
    Share 221 Tweet 138
  • Matching Buyers and Sellers in M&A

    540 shares
    Share 216 Tweet 135

CONNECT WITH US

Advertisement Banner Ad Advertisement Banner Ad Advertisement Banner Ad
Advertisement Banner Advertisement Banner Advertisement Banner

BECOME AN INSIDER

Get Financial Technology Today news and updates in your inbox.

Strategic Communications Group is a digital media company that helps business-to-business marketers drive customer demand through content marketing, content syndication, and lead identification.

Related Communities

Future Healthcare Today
Government Technology Insider
Modern Marketing Today
Retail Technology Insider
Today’s Modern Educator

Quick Links

  • Home
  • About
  • Contact Us

Become a Sponsor

Financial Technology Today offers content and advertising sponsorships to leading technology solution and service providers. Interested in becoming a sponsor? Contact us!

© 2023 Strategic Communications Group, Inc.
Privacy Policy      |      Terms of Service

No Result
View All Result
  • Home
  • About
  • Banking
  • Insurance
  • Categories
    • Digital Transformation
    • Customer Experience
    • Cybersecurity & Risk
    • Regulation & Compliance
    • Claims Management
  • Contact Us