Thursday, December 12, 2019

Handling PubSub updates from multiple clients in pub-sub scenario with Reactive Extentions

A typical pub-sub scenario is subscribing to handle requests from multiple clients and then handling client updates in a buffered way. Let's see how this could be done using Reactive Extensions.

To make this scenario more concrete - imagine your server receives a stream of frequent updates on number of stocks (e.g. IBM_1...IBM_10 in example below). We want to process the most recent price per stock and display it to a trader at most once a second.

All we would need - is to use ReactivePubSubHandler given below, providing "selector" to identify a pub-sub client (i.e. what stock was updated) and an action to be implemented (e.g. update price display) - say once in a second.



using System;
using System.Reactive.Linq;
using System.Reactive.Subjects;
using System.Threading;

namespace ReactivePubSub
{
    class Program
    {
        class ClientUpdate
        {
            public string StockName { get; set; }
            public float Price { get; set; }
        }

        static void Main()
        {
            var sampledAction = new ReactivePubSubHandler<ClientUpdate, string>(1000, _=>_.StockName, _ => Console.WriteLine(
                $"{DateTime.Now} Processed update for client #{_.StockName} Price={_.Price}"));

            var rndStockName = new Random();
            
            for (int i = 0; i < 1000; i++)
            {
                var clientUpdate = new ClientUpdate()
                {
                    StockName = "IBM_" + rndStockName.Next(0, 10),
                    Price = (float) (rndStockName.Next(10000, 11000) / 100.0)
                };
                sampledAction.OnClientRequest(clientUpdate);
                Thread.Sleep(200);
            }
        }
    }

    public class ReactivePubSubHandler<TClientData, TClientDataKey>
    {
        public ReactivePubSubHandler(int sampleIntervalMs, Func<TClientData, TClientDataKey> pubSubTopicSelector, Action<TClientData> pubSubAction)
        {
            var sampledActionsSubscription = _clientsSubscription.GroupBy(pubSubTopicSelector)
                .Select(x => x.Sample(TimeSpan.FromMilliseconds(sampleIntervalMs))).SelectMany(x => x);
            sampledActionsSubscription.Subscribe(pubSubAction);
        }

        public void OnClientRequest(TClientData clientData)
        {
            _clientsSubscription.OnNext(clientData);
        }

        private readonly Subject<TClientData> _clientsSubscription = new Subject<TClientData>();
    }
}

Wednesday, November 07, 2018

Compiling QuantLib with Visual Studio 2017 and boost 1.68

This post is a self-reminder.
Once QuantLib binaries are created as described below - you should reference .NET assembly (NQuantLib.dll) and have native .Dll (NQuantLibc.dll) to be added as a Context file with "Copy always" attribute in "Copy to Output directory".
Compiling QuantLib for Visual Studio 2017
1.     Make sure VS 2017 setup installed with Windows 8.1 SDK:

2.     Download pre-compiled boost library for VC 14 (x64 version) from https://sourceforge.net/projects/boost/files/boost-binaries/1.68.0/
  Install boost binaries boost_1_68_0-msvc-14.1-64.exe to C:\Boost\boost_1_68_0
3.    Add C:\Boost\boost_current.props file (e.g. download from https://github.com/Studiofreya/boost-build-scripts/), e.g.:


5.    Unpack SWIG to C:\SWIG\swigwin-3.0.12
6.    Add C:\SWIG\swigwin-3.0.12 to system PATH environment variable
7.    Unpack QuantLib to C:\QuantLib\QuantLib-1.13
8.    Unpack QuantLib-SWIG to C:\QuantLib\QuantLib-SWIG-1.13
9.    Modify  C:\QuantLib\QuantLib-1.13\QuantLib.props by including the following section:
   
 

10.  Enable (uncomment) QL_ENABLE_THREAD_SAFE_OBSERVER_PATTERN in ql\userconfig.hpp file
11.  Build QuantLib.sln
12.  Run the swig.cmd file located in QuantLib-SWIG\CSharp folder
13.  Modify C:\QuantLib\QuantLib-SWIG-1.13\CSharp\QuantLib.props by including the following section:

















14.  Build C:\QuantLib\QuantLib-SWIG-1.13\CSharp\QuantLib.sln
15.  Use the following residuals:
·         C:\QuantLib\QuantLib-SWIG-1.13\CSharp\cpp\bin\vc141\x64\Debug\NQuantLibc.dll
·         C:\QuantLib\QuantLib-SWIG-1.13\CSharp\cpp\bin\vc141\x64\Debug\QuantlibWrapper.pdb
·         C:\QuantLib\QuantLib-SWIG-1.13\CSharp\csharp\bin\vc141\x64\Debug\NQuantLib.dll
·         C:\QuantLib\QuantLib-SWIG-1.13\CSharp\csharp\bin\vc141\x64\Debug\NQuantLib.pdb

Monday, December 24, 2012

Monday, May 07, 2012

GC KeepAlive method–what it really does

Quite useful, I was missing this corner..
Below is quote from documentation as it is provided by code author.



// This method DOES NOT DO ANYTHING in and of itself.  It's used to
// prevent a finalizable object from losing any outstanding references
// a touch too early. The JIT is very aggressive about keeping an
// object's lifetime to as small a window as possible, to the point
// where a 'this' pointer isn't considered live in an instance method
// unless you read a value from the instance. So for finalizable
// objects that store a handle or pointer and provide a finalizer that
// cleans them up, this can cause subtle ----s with the finalizer
// thread. This isn't just about handles - it can happen with just
// about any finalizable resource.
//
// Users should insert a call to this method near the end of a
// method where they must keep an object alive for the duration of that
// method, up until this method is called. Here is an example:
//
// "...all you really need is one object with a Finalize method, and a
// second object with a Close/Dispose/Done method. Such as the following
// contrived example:
//
// class Foo {
// Stream stream = ...;
// protected void Finalize() { stream.Close(); }
// void Problem() { stream.MethodThatSpansGCs(); }
// static void Main() { new Foo().Problem(); }
// }
//
//
// In this code, Foo will be finalized in the middle of
// stream.MethodThatSpansGCs, thus closing a stream still in use."
//
// If we insert a call to GC.KeepAlive(this) at the end of Problem(), then
// Foo doesn't get finalized and the stream says open.
[MethodImplAttribute(MethodImplOptions.InternalCall)]
[ReliabilityContract(Consistency.WillNotCorruptState, Cer.Success)]
public static extern void KeepAlive(Object obj);

Sunday, April 08, 2012

Get user name in SQL

… without domain:

SELECT PARSENAME(REPLACE(ORIGINAL_LOGIN(), '\', '.'), 1)

Currencies having stocks quoted in cents

I think this is exhaustive list for now:

BWp Botswana Pula
GBp British Pound
ILs Israeli Shekel
KWd Kuwaiti Dinar
MWk Malawian Kwacha
SZl Swaziland Lilangeni
ZAr South African Rand

“267: The directory name is invalid” error when running via runas command

Should be one of:

  • User Access Control settings prevent you from executing a program
  • Directory has no access for user which is specified in runas
  • You shared your folder and that reset user rights on the folder –
    go to Properties->Security and add runas user..

Monday, January 02, 2012

SQL–how to find nth row / nth order element

Example below is self-explanatory – you should assign rank by desirable criteria and then just group by your categories (product id and name is sample below), selecting only entries of certain rank. In sample below you would select all entries with second biggest ID per category.

CREATE TABLE  #test ( id int, ProductName VARCHAR(25) )
insert into #test
select 1, 'Apple' union all
select 2, 'Apple' union all
select 5, 'Apple' union all
select 3, 'Orange' union all
select 4, 'Orange' union all
select 10, 'Orange'
SELECT * FROM #test

SELECT maxID FROM
(
    SELECT MAX(id) AS maxID, ProductName AS nn, RANK() OVER (PARTITION BY ProductName ORDER BY id DESC) AS MyRank
    FROM #test
    GROUP BY id, ProductName
) tmp
WHERE tmp.MyRank = 2

Get current username in SQL

SELECT PARSENAME(REPLACE(ORIGINAL_LOGIN(), '\', '.'), 1)

Tuesday, October 11, 2011

What is a size of DateTime type in C#?

What is a size of DateTime type in C#? - A trivial question, unexcitingly facing few obstacles. Self-explanatory code below describes how you can't get it and how you can (and yes, it's 8 bytes):
using System;

namespace DateTimeSizeExample
{
  public struct TypeSizeProxy<T>
  {
    public T PublicField;
  }

  public static class SizeCalculator
  {
    public static int SizeOf<T>()
    {
    try
    {
      return System.Runtime.InteropServices.Marshal.SizeOf(typeof(T));
    }
    catch (ArgumentException)
    {
      return System.Runtime.InteropServices.Marshal.SizeOf(new TypeSizeProxy<T>());
    }
  }

  public static int GetSize(this object obj)
  {
    return System.Runtime.InteropServices.Marshal.SizeOf(obj);
  }
}

internal class Program
{
private static void Main(string[] args)
{
// Error: 'System.DateTime' does not have a predefined size, therefore sizeof can only be used in an unsafe context
// (consider using System.Runtime.InteropServices.Marshal.SizeOf)
//int s1 = sizeof(DateTime);

// Run time Argument Exception: Type 'System.DateTime' cannot be marshaled as an unmanaged structure; no meaningful size or offset can be computed
//int s2 = System.Runtime.InteropServices.Marshal.SizeOf(typeof(DateTime));

int dateTimeSize = SizeCalculator.SizeOf<DateTime>(); // 8 bytes
}
}
}

Saturday, January 29, 2011

Dynamic casting in c#

var typesSubset = new[] { typeof(decimal), typeof(Int32), typeof(Int64) };

Type someDynamicType = typeof (int);

int someDynamicValue = 777;

if (typesSubset.Any(t => t == someDynamicType))

{

    object result = Expression.Lambda<Func<object>>(

        Expression.Convert(Expression.Add(

                               Expression.Constant(someDynamicType), Expression.Constant(someDynamicValue)),

                           typeof (object))).Compile()();

 

    Console.WriteLine(result); // 777

}

Pass xml to stored procedure

In C#:

    1 // create XML string

    2 var dataToPass = new StringBuilder("<RowsOfNumbers>");

    3 dataToPass.AppendFormat("<id>{0}</id>", 1);

    4 dataToPass.AppendFormat("<id>{0}</id>", 2);

    5 dataToPass.AppendFormat("<id>{0}</id>", 3);

    6 dataToPass.Append("</RowsOfNumbers>");

    7 

    8 // call SP

    9 myDB.ExecuteStoredProcedure("GetXmlData", new object[] { dataToPass.ToString() });

IN SQL:

CREATE PROCEDURE [dbo].[GetXmlData]    (@dataIdsXml XML)

AS

    DECLARE @dataIdsTable TABLE (ID int)

 

    INSERT INTO @dataIdsTable (ID) SELECT ParamValues.ID.value('.','VARCHAR(20)')

    FROM @dataIdsXml.nodes('/RowsOfNumbers/id') as ParamValues(ID)

 

    SELECT [ID] FROM @dataIdsTable WHERE [ID]=1

GO   

Friday, January 28, 2011

Workforce turnover around me (LinkedIn stats)

Following to LinkedIn data 57 out of my 382 contacts changed jobs in 2010.
This is 15%. Probably actual rate is a bit higher since in some cases this info may be not updated in LinkedIn.

Subset of TED lectures on health

(with bits of practical information)

http://www.ted.com/talks/gregory_petsko_on_the_coming_neurological_epidemic.html – drink coffee, no strikes/box, no avian flue, low pressure, 3 grams of omega-three a day
http://www.ted.com/talks/carl_honore_praises_slowness.html
http://www.ted.com/talks/dan_buettner_how_to_live_to_be_100.html
http://www.ted.com/talks/martin_rees_asks_is_this_our_final_century.html
http://www.ted.com/talks/dean_ornish_says_your_genes_are_not_your_fate.html
http://www.ted.com/talks/juan_enriquez_shares_mindboggling_new_science.html - current crisis and future bionic technology
http://www.ted.com/talks/julian_treasure_the_4_ways_sound_affects_us.html – well once I used nice utility (still downloadable) – Aire Freshner
http://www.ted.com/talks/mark_bittman_on_what_s_wrong_with_what_we_eat.html

Just interesting:
http://www.ted.com/talks/misha_glenny_investigates_global_crime_networks.html

Fallen giant

Few photo subsets from this blog:

“Heroic deeds live in eternity” – are they?

 

Thursday, August 26, 2010

16 years of survival in extreme conditions - in 43 sentences

Below is my (partial) translation of a short story written by Varlam Tihonovich Shalamov who was prisoner of Gulag camps from 1937 till 1953.
It’s called “What I have seen and understood in jail”.
16 years of survival in extreme conditions in 43 sentences…

What I have seen and understood in jail:
1. Extreme fragility of human culture, civilization. Man turns into beast after three weeks of hard work, coldness, hunger and beating.
2. The main way of spirit defilement is coldness – people enjailed in camps of Asia hold for longer – it was warmer there.
3. I understood that friendship and fellowship never starts in difficult, really difficult conditions – when your life is a stake. Friendship may start in conditions that are difficult but feasible.
4. I understood that emotion that man keeps for last is the emotion of hate. It’s enough flesh on a hunger man to keep only his hate. A hunger man is indifferent to the else.
…
7. I understood that humans raised the humankind because human is more strong and tenacious than any animal – horse can’t sustain hard work in conditions of Far North, human – does.
8. The only group of people that were behaving a bit like humans were religious, mostly sectarian and camisters.
9. Former militaries and politicians are broken first.
10. I’ve seen what a sound argument an ordinary slap can be.
11. Mob differentiates leaders by their ardor and strength of beating.
12. Beating is an irresistible argument.
…
16. I understood that you may live only by hate.
17. I understood that you may live by apathy.
18. In extreme conditions – man not motivated by his hopes (there is no hopes), not by his will but rather by animal instinct, instinct of survival similar to one that exist in tree, animal or a stone.
…
23. I’ve seen that women more fair and selfless than man. There was not a single case when man followed his wife to the North, but the opposite was common.
…
31. You should differentiate people not as ‘good’ and ‘bad’, but as coward and brave. 95% are coward and ready to do any villainy in case of even minor danger.
32. I’m convinced that every single hour you’ve spent in camp adds to your defilement.
…
36. I learned to plan one day ahead only
37. I understood that thieves are not humans.
…