Thursday, December 21, 2006

Ink recognition in WPF

Tablets world is coming, but we, who even have no laptop (due my manager's greed), working with workstation. Our clients want top edge technology approaches, such as geometrical forms and inks recognition within WPF code in regular desktop computer. So, let's see how we can do it. Today we'll create hand drawing form with basic geometry and handwrited text analysis

First of all, we'll need to bare all ink-related assemblies. Let's put on Microsoft.Ink and Minrosoft.Ink.Analysis. It comes with IACore and IAWinFX in "foundation world". So we'll go to C:\Program Files\Reference Assemblies\Microsoft\Tablet PC\v1.7\ and take everything from there into our application.

Next thing is to create the place where we'll draw our input. InkCanvas looking promise for me

<InkCanvas Width="Auto" Height="Auto" Name="myInkCanvas"  />

Next we'll create InkAnalyzer to analyze our inking. We'll work asyncronous to get nice performance

InkAnalyzer m_analyzer;

        void onLoaded(object sender, RoutedEventArgs e)
        {
            m_analyzer = new InkAnalyzer();
            m_analyzer.AnalysisModes = AnalysisModes.AutomaticReconciliationEnabled;
            m_analyzer.ResultsUpdated += new ResultsUpdatedEventHandler(m_analyzer_ResultsUpdated);
        }

Fine. Now we should provide to our InkAnalyzer information about what it'll going to work for. So, start collection and erasing strokes will make my live easier. We'll have to tell background engine of ink analysis when we want to do anything. Sure, we'll do it after new stroke arrived.

void onStrokeErasing(object sender, InkCanvasStrokeErasingEventArgs e)
        {
            m_analyzer.RemoveStroke(e.Stroke);
        }
 
 
 
        void onStrokeCollected(object sender, InkCanvasStrokeCollectedEventArgs e)
        {
            m_analyzer.AddStroke(e.Stroke);
            m_analyzer.BackgroundAnalyze();
        }

 


Goody. Now let's see what we have. We can draw geometry and we can write text. Let's start with the text. We'll create little helper to draw our stings on the surface

class GistaFigure:FrameworkElement
    {
        FormattedText ft;
        Point cent;
        private GistaFigure() { }
        public GistaFigure(string name, Point center, double size, Brush color)
        {
            init(name, center, color, size);
        }
        public GistaFigure(string name, Point center, Brush color)
        {
            init(name,center,color, 12);
 
        }
 
        void init(string name, Point center, Brush color, double size)
        { 
            ft = new FormattedText(
                name, System.Globalization.CultureInfo.CurrentCulture,
               FlowDirection.LeftToRight,
               new Typeface(new FontFamily(), FontStyles.Normal, FontWeights.Normal, FontStretches.Normal), size, color);
 
 
            cent = center;
        }
 
        protected override void OnRender(DrawingContext drawingContext)
        {
            base.OnRender(drawingContext);
            cent.X -= ft.Width / 2;
            cent.Y -= ft.Height / 2;
            drawingContext.DrawText(ft,cent);
 
        }
    }

Now we can analyze it. I want to color my result other colors, related to confidence of the detection

void m_analyzer_ResultsUpdated(object sender, ResultsUpdatedEventArgs e)
        {
            if (e.Status.Successful)
            {
                ContextNodeCollection nodes = ((InkAnalyzer)sender).FindLeafNodes();
                foreach (ContextNode node in nodes)
                {
                    if (node is InkWordNode)
                    {
                        InkWordNode t = node as InkWordNode;
                        Rect l = t.Location.GetBounds();
                        Point a = new Point(l.Left + l.Width / 2, l.Top + l.Height / 2);
                        double de = l.Height;
                        Brush be = Brushes.Blue;
 
                        switch (t.InkRecognitionConfidence)
                        { 
                            case InkRecognitionConfidence.Intermediate:
                                be = Brushes.Green;
                                break;
                            case InkRecognitionConfidence.Poor:
                                be = Brushes.Red;
                                break;
                            case InkRecognitionConfidence.Unknown:
                                be = Brushes.Brown;
                                break;
                        }
                        GistaFigure figure = new GistaFigure(t.GetRecognizedString(), a,de , be);
                        myInkCanvas.Children.Add(figure);
                    }

Guta! We have "HELLO" recognized, but I want to recognize geometry. Else, my client will throw me out 'cos the way I'm writing text really horrible with the mouse (I have no tablet, you remember?). Let's recognize geometry. Those are other nodes, named InkDrawingNodes. Do it

else if (node is InkDrawingNode)
                    {
                        InkDrawingNode d = node as InkDrawingNode;
                        GistaFigure figure = new GistaFigure(d.GetShapeName(), d.Centroid, Brushes.Red);
                        Shape shape = d.GetShape();
                        if (shape != null)
                        {
                            shape.Stroke = Brushes.Blue;
                            shape.StrokeThickness = 2;
                            myInkCanvas.Children.Add(shape);
                        }
                        myInkCanvas.Children.Add(figure);
                    }

Donno. I have nothing more to add here. So let's draw (for those, who know to do it with only mouse)


Source code for this article

7 comments:

Qyoong said...

I downloaded your source code for Ink recognition in WPF, very thank you.
I can test on its object "InkAnalysis.exe", it's work. But I cannot open its project/solution. It told me that "The project type is not supported by this installation.

How can I solve this problem?
I think I have to download something new but I don't know which one. Could you please tell me about this?

I'm using Windows Vista Home Premium, MS Visual Studio 2005, already install silverlight.

Thank you

Tamir Khason said...

You should install WPF tools for VS2005 in order to make possible to open this solution from Visual Studio

Qyoong said...

Thank you,

I tried to find the installer (I know I have to install WPF because vs2005 doesn't support) but I don't know which one I have to select to install.

Anyway, I will keep try to find out what I have to install at least what I think is all right.

Really thank you

Qyoong said...

Finally, I got it. Thank you for your instruction.

Actually, I'm working on a project (handwriting recognition). Before, I know about Ink Recognition using Silverlight technology, I didn't know anythink until I know about this. So, now I'm trying to learn from source code such as from yours.

If you have any recommendation, please do me a favour to let me know.

Thank you

Unknown said...

Interesting stuff.

Do you have any idea where to find good resources on the maths involved in geometric shape recognition?

It's lovely to have an API that lets you say 'what shape is this', but I'd really like to know how this actually works under the hood.

Unknown said...

Wow! Thank you! Honestly, I love the internet for having writers like yourself that sit down and take time to educate, and mentor us into preparing us through the right way to succeed in life. Whether you think that this helps or not, I just want you to know that it most definitely makes a huge impact and a difference for someone like myself. I'm taking my time to write this because I don't want you to go feeling un-noticed! You have most definitely been recognized and you should know that this has surely helped me along my way! Thank you once again, really..
Have a great day!
Samsung CLP-600 toner

Jim Ikuta said...

This is so great! Thank you! Looking forward to see more
Canon MF743Cdw Toner