Saturday, April 18, 2009

How to re-recover your files (mp3, etc...) after recovering from a bad damage

The recovery mentioned above means to crop out the excessive bits added to the begining of the file, probably after a system recovery. This was a true experience, that I had about an year back. I had a hard disk crash, so that all the data is gone. Finally I was able to recover the files, but with some arbitrary size of bits appended front of some (majority of) files.
There is a simple and more straight forward solution. That is to use a hex editor, and open each file with the editor, then delete the junk bits and save the file. It will be hell a lot of pain, if you are going to follow above method, since there may be several thousands of files to be edited. (Actually in the first place, I was determined to follow this method. Eventually I realised the silliness after couple of days :D)
Therefore I came up with a simple C++ programme to get the job done. I thought of sharing it with you, so that it may help you out in some case :). The programme was implemented to be used in Windows platform, but it could be worked out to run on UNIX without much of a pain.
In the current version, the executable should be moved to the directory that contains the set of corrupted files and directories containing them. Then run the programme using command prompt. It will recover the files in the current directory as well as iteratively the sub-directories.
The programme is written in such a way that it will remove all the bits before the file type's header tag. In this case it will cure only one file type at a run. But feel free to change the code according to your wish and to make your own recovery programme.
You can get the source code from here

Thursday, April 16, 2009

High Performance Computing for Financial Mathematics (HPC4Finance)

Most of the mathematical models in finance are highly resource intensive, when it comes to processing. Therefore, it is encouraging to go for a High Performance Computing resolution. As a result, We, Damitha, Nilendra, Thilina and myself (Waruna) came up with a system which consists of a computational cluster, database cluster and an Excel plug-in to speed-up the financial calculations as well as to add more user convenience by providing a user interface as an Excel plug-in. Hence your results are only a click away.

The system is developed in such a way that other financial models can be integrated to the system dynamically by the user (in this case the developers). In the latest release of HPC4Finance has inbuilt support for three (3) short rate models, namely Geometric Brownian Motion (GBM), Vasicek Model and Cox-Ingersoll-Ross model (CIR).

MPICH2 (1.0.8) cluster which is an implementation of MPI 2.0 specification, is used to implement the computational cluster. The cluster is built on UNIX environment. But it is available for other platforms as well (i.e. Windows even though its not tested enough). The two major problems, we faced with the MPICH2 cluster are load balancing and the fault tolerance. A simple resolution is introduced to improve load balancing by using a round robin algorithm to resolve the jobs (tasks) and their respective execution nodes. Say, that 2 jobs where both are supposed to
run in 10 machines (nodes), are submitted to the cluster where there are 20 machines. Then, in default case, both the jobs will run in the same set of nodes and there are 10 nodes idling. This is a huge waste of resources. Therefore, as our resolution, the jobs are explicitly submitted to the next (available) set of nodes in a round robin manner. Unfortunately, we could not come up with an straight forward mechanism to fault tolerance and fail over management which should be a functionality of the cluster itself. But we provide a logging system which enables to track the fault and restore or restart the job.

MySQL cluster is used to implement the database cluster. The data is automatically replicated through the cluster. The cluster support data node fail-over,
but not SQL (API) node fail-over.

Excel plug-in is the most important component which adds a great deal of value to the system. It enables financial professionals who are not keen in programming and computer clusters to run their financial models by using just a few mouse clicks. The results are drawn on a graph and presented to the user with the other relevant information. The plug-in enables users to import and update financial data through a web service or form a .csv file.


The user is also facilitated with a ribbon, to change and the set the preferences and settings of the plug-in.
The user also equiped with ability to stop (kill) jobs any time which enables users to terminate jobs whose parameters are entered incorrectly.





The system (HPC4Finance) is just an insight to the possibilities of High Performance Computing to the field of Finance with great deal of extendibility and user convenience.

Reference

T.P. Dampahala, H.D.D.D. Premadasa, P.W.W. Ranasinghe, J.N.P. Weerasinghe, K.A.D.N.K. Wimalawarne, "Efficient High Performance Computing Framework for Short Rate Models," ams, pp.608-613, 2009 Third Asia International Conference on Modelling & Simulation, 2009

Can be found at -
Efficient High Performance Computing Framework for Short Rate Models - http://doi.ieeecomputersociety.org/10.1109/AMS.2009.27

List of published papers in
the 3rd Asia International Conference on Modeling & Simulation