Principles of billing
This section describes the rules and principles that are related to billing. In OSB the billing process has two major components, maintenance of a balance sheet for each contract and generation of invoices based on the balance sheets.
The billing process also could be described this way: In OSB charges are permanently collected and written to a balance sheet. Invoice generation groups the charges on the balance sheets, calculates taxes and determines the GL accounts.
In its current state, the document is an unstructured collection that reflects the outcome of our discussions.
This chapter describes the workflow for the generation of one invoice. For each contract the OSB library creates one invoice. The billing application is responsible to determine which contracts should be invoiced and to distribute the workload among the available billing servers.
Diagram I shows the activities of the billing application:
Diagram I: Activity of billing application
This section outlines the multithreaded model for the billing application that takes care of batch invoicing. viz: preparing of invoices at the end of the billing period. For example: end of the month. In batch invoicing the focus is on application efficiency, where the requirement is to create the maximum number of invoices in a given time. Our preliminary analysis for a highly adaptive batch invoicing application indicates that multithreading can be used as a mechanism to maximise productivity by minimising the amount of time the CPU will remain idle while the program waits for resources. The application will use POSIX threads. Most Unix vendors have adopted the POSIX standards. So programs written uing the POSIX standard is portable to different Unix platforms.
The application works as follows. The application spawns mainly
two sets of threads: One populator thread and multiple invoicer
threads. The populator thread prepares the data for the contracts
to be invoiced and the invoicer threads works on the data and
prepares the invoice. Each invoicer thread, after completing each
individual invoicing, works on the next contract which is in the
queue, until the commit size is reached when it commits them into
the database. After commit, the thread continues with the
invoicing. Data communication between the populator threads and the
invoicer threads is achieved by maintaining data objects visible to
both the threads.
The application hence uses the producer-consumer model. Another thread spawned by the application takes care of application logging.
The contracts to be invoiced are maintained in a collection
object located at the billing runtime. The populator thread writes
the available contracts to the list from where the invoicer threads
retrieves them for invoicing. The list is maintained at a
predefined maximum size. When the size is reached the populator
thread stops adding contracts to the list, until a condition is
reached where the list is 'almost' empty. This will be a
configurable parameter, which can be set in a configuration file.
When the worklist is almost empty it ensues a signal to the
populator thread. The populator thread which will be in a waiting
condition again starts populating the list.
Similar to the behaviour of the contractor threads, the invoicer
threads too go to a waiting state when it finds the contract list
empty. The threads remains in this state until a signal is issued
by the worklist about an event of another contract added to the
The applications spawns a logger thread that takes care of the
logging process. This is done to increase application efficiency.
Since file operations consumes valuable time, it is advisable not
to give this task to the populator and invoicer threads. The
populator and invoicer threads write the log data to a log-list
object, which is written to the file by the logger thread.
Thread synchronization is achieved using mutexes. When one
thread is about to access the resource, a lock is applied on the
list which is unlocked only after the read/write operation by the
thread is complete. All other threads will have to wait till
The threads communicate with one another mainly for data transfer between them and ensuing signals between them. Most of these communication can be achieved by maintaining global variables. The threads writes the status to these variables while others read. Figure shows the possible tracks of communication between the threads.
Inorder to maximise the efficiency of the application, the
generated invoices will not be committed to the database one at a
time. This is to avoid the overhead of performing a database write
for each invoice. The solution is to commit in batches. So a number
of generated invoices are committed into the database en bloc. The
commit size will be a configurable parameter for the user of this
application. Alternatively if there are no more contracts to be
invoiced available in the list, the invoicer thread commits the
invoices already processed.
When the populator has finished adding contracts to the list, it notifies the main. Main notifies this to the invoicer and logger threads and ensues a wake-up call for any sleeping threads. The invoicer threads exits when it finds that there is no more work left in the list. The logger also behaves in a similar fashion.
Diagram II shows the steps for the invoice generation of one contract:
|Per default taxes are computed after marketing. This means that
any amounts generated during marketing are considered for
If marketing needs gross amounts, taxes should be computed before and afterwards.
On the other hand if amounts generated by marketing are never taxable, there is no need to compute taxes afterwards.
Hint: This is one reason why OSB applications are customer specific!
This chapter discusses the possible ways of storing the generated invoices (xml files).
Currently, there are mainly two ways to store all the invoices:
In the first solution, all the newly generated invoices will be kept in several directories. The purpose of using several directories is due to the limitation of the directory mechanism in Unix system. The more file entries in a directory (the directory size), the slower it is to access it. The directory structure will be organised as shown in diagram III:
Diagram III shows that the system will keep a record of invoices for 12 months. In the actual practice, this may not be the case.
In the implementation, for the starting of every month, the month directory must be removed. (e.g. before generating the invoices for the month March, the system will have to remove the "March directory" recursively.) After that, a new "March directory" will be created and the invoices can now be generated. The removal of old "month directory" is needed because the directory size will never shrink even if the system has removed all file entries in it.
Got to determine the total of invoices that will probably be generated each time in order to estimate the total subdirectories (under the monthly directories) needed to implement this approach. Maximum file entries in each directory must also be determined.
Second solution is to store all the invoices into a database (or at least handle by database). Target DBMS in this solution is the Oracle8i. This database solution can be further refined into 3 different approaches:
In this first approach, all the invoices (xml files) will be processed by the DBMS. All the elements (data) will be extracted from the xml files and inserted into tables. The structure of the xml file will also be saved into the DBMS. Because all the data has been put into tables, all actions (e.g. query, update, search etc.) can be performed on the xml data just as ordinary database data. On the other hand, because all data has been extracted, all xml files will be discarded. As a result, the output xml files from the database may be different from the original copies (the sequence of the data). Another problem is that all the comments will be lost.
Currently, there's no requirement on updating the invoices (xml files) generated. So, the benefit of this approach may not be applicable to the system.
In this second approach, each and every invoices (xml files) will be stored into the database as a CLOB object. In another word, the whole file will be stored as it is (a sequence of characters). With this approach, the original files will be kept intact together with its structures, data sequence, and comments. For this state, this approach has fulfilled its aim of storing the invoices. If actions like queries, updates, searches are needed, then the "InterMedia Text Indexing" feature provided by the Oracle8i can be used. This indexing method can perform certain parsing action on the invoices (xml files), gather the needed information, and store them in the database. Thus, queries regarding xml tags and sections can be performed.
If compare, queries on CLOB data type may not be as fast as the object-relational storage approach. But again, this is currently not an issue to our system. Furthermore, this approach do shorten the output time of the invoices (xml files) from the database.
In this last approach, the DBMS does not actually keep the invoices (xml files) in the database. All it does is to keep the invoices in the file system and create locator objects, in the columns of the tables, to point to the actual files. This approach can also implement the "InterMedia Text Indexing" to enable queries, etc. Because the invoices is still intact, all structure, data sequence and comments are preserved. A great benefit from this approach is that, the database size won't get too big.
The internal implementation of the Oracle8i DBMS:
From the understanding of the implementation stated above, the invoices can be accessed directly without going through the database.
Invoices need to be kept for future references. No matter which solution from the above being selected, the storage size issue still exist. In order to counter this problem, compressing invoices (xml files) seems to be a possible way. Currently, there exist a C library named zlib , that can be used to perform compression as the invoices are being generated.
Although this approach do benefit the system from saving up more then 50% (claimed by the zlib's developers) of storage space, it make parsing those invoices impossible (without first umcompress it). In certain cases, for example, the customer care services, certain specific invoices may be requested by the customer. In order to get the correct invoices, every invoices will need to be uncompressed, read, compressed again until the system found the invoices it needs. In order to counter this problem, the database solution of Using BFILE data type can be of great help. Before the invoices have been compressed, they will be parsed to collect some key information such as date, customer_id, etc. Those information will be kept in the database along with the locator which points to the specific compressed invoice (xml files). This approach, can minimize the number of invoices need to be uncompressed before the needed invoices to be found.
Key information to be extracted will need to be further refine.
Base on the reseach stated above, the best solution will be a combination of
When the system is generating the invoices, key information will be captured and stored into database tables. A specially designed directory structure will be created to keep all the invoices generated. Before the invoices have been written to a file, they will be compressed. Finally, each invoice will have a locator (pointer) pointed from within a database record (row).
This chapter details with the handling of usage records. The flow is made up of the read records and process records as detailed below. We might have a sort usage records phase introduced after reading of records.
The records to be transferred from the database would be
normally in the sorted order of callId and recordId. But there
could be a possibility of a different sort requirement. This has to
be handled in the Application level.
The criteria to get the usage records needs to be specified. (ContractId for which the records need to be billed).
The process records stage parses all the usage records read. The charge traits details are identified by referencing the chargeTraitsId that is available in the usage record. A new UsageItem is created during this process. A section Id is defined for the usage record. This section Id is used to display the usage details in the invoice in a predefined order.
E.g. To List all the international calls, the national calls followed by the local calls for that particular contractId.
The above process results in a set of UsageItems and corresponding UsageDetails to be added to the invoice.
A usage record is the basic information which contains the following details required to generate an invoice.
Apart from the above the following details can be used for information purposes which is displayed on the invoice
This is Illustrated in the diagram VI.
A high level view of the UsageRecord is illustrated in the diagram VII.
A few points need to be noted at this stage
OSB creates an invoice for each contract. For each contract OSB maintains one balance sheet, that stores all charges of the next invoice. This information allows to define a straightforward principle for the billing of one contract: The billing application reads all charges from the balance sheet, calculates the taxes, determines the GL accounts, forwards the relevant information to the accounting interface (see below) and converts the invoice data into the generic format which is stored in the database.
The billing module writes the generated invoices and their details to an accounting interface. The accounting system is then responsible to handle payments and all other financial transactions.
OSB shows open invoices, received payments and the current balance of a customer on the invoice. The accounting interface should provide appropriate functions to retrieve this information.
|The original idea to write all financial transactions to the balance sheet was dismissed: Transactions handled by an external finance system should be written to an OSB interface that updates the balance sheet accordingly. This requirement can not be fulfilled in all cases. But this would mean that OSB has to support two different concepts how to retrieve financial data about a customer and contract.|
OSB knows three kind of charges: subscription, usage and one-time charges. Charges can only be risen on the basis of a personalized product that is assigned to a contract. The definition of the charges, i.e., how much an item costs and how it is charged, is linked to the product or its product items.
Diagram IV shows the status diagram for a personalized product and personalized product item. For each status OSB allows to charge subscription fees for the time period a status was valid for a personalized product and personalized product item.
One-time charges are billed every time they are assigned to a
contract. Of course there must be a reason for such an assignment:
This can be either a status change, an OSB event or manual
(You might think of yet another reason to raise a one-time charge: The simple fact that a given one-time charge has never been assigned to a contract before. Technically this is the worst of all reasons because we have to remember that the charge was billed as long as the contract exists.)
Events are risen automatically by OSB in well defined
situations. Examples for such events are: Itemized bill calculated,
replacement of a resource, change of address, dunning of an
invoice, ... only to name a few.
Not all events are directly related to one product of a contract: an invoice, e.g., is generated for all products of a contract. Because there must be a way to define which charges should be applied in such situations, the terms and conditions valid for a contract should include a reference to a product that contains the necessary information.
Status changes (see diagram xxx) are very similar to events. We are using a different term because a status changes always refers to a personalized product or a personalized product item.
Usage charges for a call are calculated during rating by the
RatingRule that is valid for the TariffClass and TariffPeriod of that call.
The billing process is responsible to determine the GL account of the charge, possibly apply rounding and to calculate any taxes. All three items are determined from the ChargeTraits that can be configured per TariffClass and TariffPeriod in a TariffSystem. For each call processed by rating this information shoud be stored in the gnerated usage records to avoid recomputation of the criteria during billing time.
The next part of this chapter describes the configuration of
charges and later down we'll discuss how the charges that apply for
a personalized product are determined.
Details about the configuration of usage charges are (or will be :-) described in the class tariffSystem. They are not discussed any further in this chapter.
In this subchapter we discuss the configuration of subscription and one-time charges and how we link event to one-time charges.
There is no difference in the charge configuration of products and product items with one exception. This is related to the fact that if a product is assigned to a contract, OSB will always create a personalized product. For the product items however a personalized product item is created only if individual configuration per contract is needed, while other product items are inherited by the personalized product. We'll discuss the details further down.
Within this chapter I'll use product to refer to a product or product item.
For each Product and ProductItem charges are configured separately as shown in diagram V:
The charge configuration is encapsulated in the class PriceList.
This has the advantage that we are free to allow several products and/or product items to use the same charge configuration, say same instance, of PriceList. This however would not only require to maintain a status for a price list but we also would need functionality to verify that a price list change is permissible for all products and/or items using this price list.
The subscription charges are calculated by subscription functions, that implement the rules how to calculate a product's subscription fee. Examples for subscription functions are:
A subscription function processes the status changes of a
personalized product during the invoiced period and -if applicable-
generates a list of one or more Amount.
Subscription functions can be parametrized, e.g., the time period which is the basis for proration or which states should be charged in advance. The number of needed subscription functions is greatly reduced by the possibility of their configuration
The approach of offering several subscription functions has 2 major advantages:
For each status that may be charged we need the charge and the
ChargeTraits: The former is used by the
subscription function to calculate fee for the status and the
latter determine Amount.traits what
allows separate bookkeeping for each product (or product item) and
Additional attributes may be needed to define the charges of each status. Such attributes however should never be specific to particular subscription functions: Such a design inevitably would mix the responiblities. The correct solution is to add such attributes to parametrization list of the subscription function.
|As of 18-Feb-01 it is not yet decided if we allow for different currencies for the subscription fees of a product. Technically there is no problem, but what if we want to charge the more expensive state if several states apply for a given day and personalized product?|
You might ask why OSB does not allow to define a subscription function for each status. We have in fact discussed this issue quite a long time and finally abandoned the concept: There must always be a function that coordinates the various applicable states over the whole invoice period. Such a function must be either universal or it must be selectable during configuration. In the first case it is not sure that all requirements are covered while in the second case configuration gets more complicated than necessary.
One-time charges are much easier to handle than subscription fees: Either they are charged or they are not. There is however a small difference. For some one-time charges, e.g., late fees the charged amount is a percentage of another amount. This means that we have to maintain a list of OSB events that are related to another mount and what exactly this other amount is, for late fees this would the amount of the dunned invoice.
The only information needed for billing of one-time charges is the amount and the ChargeTraits. Eventually we'll need some more settings, mainly to support configuration and on-line applications, e.g, it should be possible to flag a one-time charge as being applicable for status changes, OSB events and/or manual assignment. (Other info to be detected during design and implementation :-).
Status changes and OSB events do not differ in the way they are linked to one-time charges. For manual assignment no configuration is needed: the on-line application displays the list with the one-time charges that are configured for a product (or product item!) and the user makes his or her selection.
OSB events and status changes may be linked to one or more
one-time charges during the configuration of a product.
Once the product is assigned to a contract and an event is risen, OSB generates the one-time charges associated with that event and writes them to balance sheet of the contract. In the second step billing determines the correct GL account, rounding and taxation for the one-time charge.
For computation of charges we first need to know if a personalized product or personalized product item should be considered at all for billing or not. In the second step we have to find the status changes that are applicable and to determine for which time period a status was valid.
Diagram VII shows an example of a product P that is assigned to a contract C. P contains 2 product items: cfgPi needs individual configuration (personalization) for each contract that is not needed for noCfgPi.
After the product P is assigned to
contract C we will have the following
situation: C contains the
personalized product PP that refers
to P and -because personalization is
needed for cfgPi- contains a
personalized product item. The latter refers to cfgPi and holds the actual configuration for
No configuration is needed for the product item noCfgPi, this means that no personalized product item exists within PP.
During billing of contract C we have to answer the question whether the product item noCfgPi should be charged or not, or -more precise- if the price list pincPl is applicable. The problem actually is not only related to billing, it also applies for customer care, provisioning and rating as well. Assuming that noCfgPi contains all GSM call forwarding services the following questions are typical: "Is call forwarding available for contract C?", " During provision of C, do we have to include the call forwarding services also?" and "Is it allowable that we receive call forwarding records for contract C?".
Note that for a product the problem never exists: A product is either assigned to the contract and charged during billing or it is not assigned and therefore not available and not chargeable to the contract.
For a product item the problem is more complicated, diagram VII does actually not provide enough information about noCfgPi to answer the above questions. Remember that every product item has an attribute mandatory and may be configurable. With this we can setup the following decision matrix:
|ProductPart for contract|
|always||no||if choosen||if PPI exists||if PPI exists|
|never||no||if choosen||if PPI exists||if PPI exists|
|optional||no||if choosen (configured or not)||if PPI exists||if PPI exists|
In table I the first two columns, configuration and mandatory, define if a product part needs configuration and whether it is mandatory or not. For each combination of these two ProductPart attributes, the third column defines if OSB needs to create a PersonalizedProductItem, and the last two columns show if the product part is available and chargeable for the contract.
From table I we can see immediately that -as it is common sense-
whenever a ProductItem is usable for a
contract, it is charged as well.
This means that we have to provide the rules under which conditions a product iem is available for a contract. Table I shows that we can use the mandatory attribute of the product item and the existence of a personalized product item as criterium.
Until now we have not considered that product items may themselves consist of other product items. Again it is common sense that the parts of a product item are available (and chargeable) for a contract only if the product item itself is usable for the contract. The decision matrix for the child product items is given in table II (the result is the same as in table I):
for child created?
|yes||always||no||if PPI exists||if choosen|
|yes||never||no||if PPI exists||if choosen|
|yes||optional||no||if PPI exists||if choosen (configured or not)|
Using all information from above we are now able to define the rule when a ProductItem can be used and therefore is chargeable to the contract. The result is surprisingly simple:
|For a given Contract
C and a Product P assigned to
C (i.e. a PesonalizedProduct for the contract C and product P
exists), a ProductItem PI which is part of P, is available and chargeable to C if and only if
The parent of a ProductItem is either the Product P or a part of P
|check here for current status|
|check here for current status|
On an invoice individual charges are usually grouped with subtotals:
Usage charges Call 1 1.07 ... ... Call x 2.45 Total usage 27.85 Subscription charges Telephony 12.50 Internet Access 5.75 Total subscription 18.25 Total net 46.10 VAT 10% 4.61 Rounding -0.01 Total invoice 50.70
Unfortunately things behind the scene are more complicated as indicated by the resulting invoice:
The class Subtotal is designed to
overcome the problems described above. Subtotal is a collection of references to
individual item or other subtotals.
Instead of trying to coordinate charge changes and the calculation of subtotals the sum of the items is calculated whenever needed.
Internally, the OSB billing module works with amounts in a configurable internal precision. Standard mathematical rounding is used to determine such amounts.
Each Amount has a rounding rule,
which is used to determine net amount and rounding from the
Additionally, each Amount also has a format attribute which defines how the net amount should be formatted.
For example, assume the precision is 5 digits, rounding rule is "round to the nearest 5 cents" and the format is "Fr. %.2f":
|original amount||internal representation||net amount + rounding||formatted amount|
|1/6||0.16667||0.15 + 0.01667||Fr. 0.15|
Should we allow for a rounding rule together with the global precision?
Tax calculation consists of the following two steps:
1) For each invoice item, find the applicable taxes
2) Calculate monetary tax amounts
In the first step we use tax keys to identify a set of applicable taxes. In the second step, the taxes are calculated and rounded one by one.
Tax keys are assigned to associates, contracts and product items. For each invoice item, the combination of these three taxation keys is evaluated to determine which set of taxes is applicable. There exists a "catch all" or "wildcard" taxation key "*" which matches any value in the triplett that is evaluated. The best match is determined using different priorities for matching individual taxation keys: a matching associate taxation key has a higher priority than a matching contract taxation key which has itself a higher priority than a matching product item taxation key. The least priority is a "wildcard" match.
Associate taxation keys typically include a "regular" entry for associates who pay normal taxes and an "exempt" entry for associates who are exempt from paying taxes. Contract taxation keys may include "private" and "business" to account for different taxes for private contracts and business contracts. On the product item, there may be individual entries for the various applicable taxes and entries that refer to combinations of taxes, e.g., a "federal tax", a "state tax" and an "all taxes" entry which would include both, the federal and state taxes. Without "wildcards", these 2 associate taxation keys, 2 contract taxation keys and 3 product item taxation keys result in 12 different combinations for the taxation key triplett:
|associate taxation key||contract taxation key||product item taxation key|
Each of these tripletts would determine a set of actual taxes to be applied to the product item in question (e.g., 10% VAT for the triplett <regular><private><federal tax>). Assuming that whenever an associate is exempt from tax, then he does not need to pay any taxes, independent of the contract or product item, then we can use wildcards for a simpler setup:
|associate taxation key||contract taxation key||product item taxation key|
With this configuration, the tax determined for any taxation key triplett with an "exempt" associate taxation key is the tax linked to the <exempt><*><*> entry in the table above, e.g., a 0% tax.
Assume that even a tax exempt associate still has to pay a flat state tax, which is always, e.g., 5%. It is enough to add one entry <exempt><*><state tax> to the configuration and link it to the appropriate 5% tax. When determining the best match, we need to consider the priorities set out above. Accordingly a key triplett <exempt><private><state tax> will match the new <exempt><*><state tax> rather than <exempt><*><*> because the matching "state tax" has a higher priority than the wildcard match for the product item taxation key.
|associate taxation key||contract taxation key||product item taxation key|
Implementation hint: taxation keys can be integers (e.g., wildcard=0, regular keys = 1,2,3,...)
The result of a search for applicable taxes with a key
triplett should be a list of taxes with at least one item.
Each of these items must then be applied to calculate the tax amount per tax, round it and assign accounting information to it. The taxRule contains the logic for this calculation.
This chapter details with definition of classes involved in the usage records.
The UsageRecords for a given Contract are read from the OSB database and the UsageRecordList is populated
Criteria for retrieving the records would be the particular contract. All calls not billed since last time should be billed. Assuming that for each call record there would a flag indicating if the record is yet to be billed on not.
Once the UsageRecordList is created, the List needs to be sorted. The sort criteria would be on the CallId and RecordId (could be changed later on).
For every UsageItem, it is mandatory that at least one UsageDetail should be present.
The relationship is as shown below:
Once the UsageItems and their corresponding UsageDetails are created, they can be used to generate the Invoice.