Turning to the dark side of inking = UnprocessedInput

Sebastien Bovo · ‎May 06 2020

We learned how to draw on an InkCanvas and 'convert' the lines drawn into nice polished XAML lines with the previous article Use the UWP Inking platform as input for advanced scenarios. Let's go further and implement a "cut the lines" functionality in order to be able to divide a line in two lines like we would do using a scissor ✂.

Initial code for handling inking and creating XAML Lines

For the starting point, we take the sample code related to the article we just mentioned before.

As a reminder, we have two canvas:

One InkCanvas for accepting the drawings.
One Canvas for the resulting XAML Shapes.

<InkCanvas x:Name="inkCanvas" />
<Canvas x:Name="ShapesCanvas" />

The code behind for 'converting' strokes is displayed below. When the user finishes to draw a line (StrokesCollected event), we create a XAML Line with the same origin and end as the stroke. Then, we add this XAML line to the canvas and remove the stroke:

public MainPage()
{
   this.InitializeComponent();

   // Initialize the InkCanvas
   inkCanvas.InkPresenter.InputDeviceTypes =
      Windows.UI.Core.CoreInputDeviceTypes.Mouse |
      Windows.UI.Core.CoreInputDeviceTypes.Pen |
      Windows.UI.Core.CoreInputDeviceTypes.Touch;

   // When the user finished to draw something on the InkCanvas
   inkCanvas.InkPresenter.StrokesCollected += InkPresenter_StrokesCollected;
}

private void InkPresenter_StrokesCollected(
   Windows.UI.Input.Inking.InkPresenter sender,
   Windows.UI.Input.Inking.InkStrokesCollectedEventArgs args)
{
   InkStroke stroke = inkCanvas.InkPresenter.StrokeContainer.GetStrokes().Last();

   Line line = ConvertStrokeToXAMLLine(stroke);
   ShapesCanvas.Children.Add(line);

   // We delete the InkStroke from the InkCanvas
   stroke.Selected = true;
   inkCanvas.InkPresenter.StrokeContainer.DeleteSelected();
}

private Line ConvertStrokeToXAMLLine(InkStroke stroke)
{
   var line = new Line();
   line.Stroke = new SolidColorBrush(Windows.UI.Colors.Green);
   line.StrokeThickness = 6;
   // The origin = (X1, Y1)
   line.X1 = stroke.GetInkPoints().First().Position.X;
   line.Y1 = stroke.GetInkPoints().First().Position.Y;
   // The end = (X2, Y2)
   line.X2 = stroke.GetInkPoints().Last().Position.X;
   line.Y2 = stroke.GetInkPoints().Last().Position.Y;

   return line;
}

Here is the initial application:

Initial windows displaying strokes converted in XAML lines

We can start now!

The idea: Use the inking input to be able to 'cut' lines

We have the InkCanvas there, ready to accept pen/mouse/touch inputs. It would be great if we could leverage on it to accept interaction for drawing our 'cutting line'. The InkCanvas allows us to do it with UnprocessedInput.

The idea is to allow the InkCanvas to take input (we would like the user to draw a line to cut the XAML lines) but we do not flow this input to the InkPresenter to display the default digital ink. Instead, this input is handled only by our code.

Let's first modify the XAML in order to add a toggle switch to change the input mode between "allow inking" and "cut lines". To do so, we add a grid with two lines. The first line takes the ToogleSwitch control and adapts its height based on the height of the button. The second line takes all the remaining space of the window and shows the two canvas (Let's put the LightGray background color for this line):

<Grid>
	<Grid.RowDefinitions>
		<RowDefinition Height="Auto"></RowDefinition>
		<RowDefinition></RowDefinition>
	</Grid.RowDefinitions>


	<Grid Grid.Row="0">
		<ToggleSwitch x:Name="cutAShapeToggleSwitch"
			OffContent="Cut lines Off"
			OnContent="Cut lines On"
			Toggled="ToggleSwitch_Toggled"/>
	</Grid>


	<Grid Grid.Row="1" Background="LightGray">

		<Canvas x:Name="ShapesCanvas" />
		<InkCanvas x:Name="inkCanvas" />
	</Grid>
</Grid>

The ToggleSwitch is 'off' by default with the OffContent text displayed. If we toggle the control, it will be 'on' and will display the OnContent text. And so on... We get the toggle's changes in the code behind with the ToggleSwitch_Toggled event handler:

private void ToggleSwitch_Toggled(object sender, RoutedEventArgs e)
{
   ToggleSwitch toggleSwitch = sender as ToggleSwitch;

   if (toggleSwitch.IsOn == true)
   {
      // We go to the "cut the lines" mode
   }
   else
   {
      // We go back to normal
   }
}

UnprocessedInput unleashed

How can we switch to the "cut the lines" mode and 'take the control' of the inputs? Simply by modifying the inking processing behavior and not accept inputs for inking or erasing. This is done by modifying the InputProcessingConfiguration property of the InkPresenter:

inkCanvas.InkPresenter.InputProcessingConfiguration.Mode = InkInputProcessingMode.None;

Now, as the inking is not processed anymore, we have to do it with our code. We have three events to handle for this:

PointerPressed - Every drawing starts with this event. It allows us to execute an initialization code when the user start to draw. Typically, we initiate a XAML Shapes that we will modify for every movement of the pen/touch/mouse.
PointerMoved - This event is raised when the user is drawing on the InkCanvas. We can then handle the input and draw or modify some XAML Shapes to follow the input's changes for example.
PointerReleased - When the user finished the drawing i.e. when the device surface is not touched anymore, we consider the drawing finished. We can decide to perform actions like selecting/modifying shapes or launch a shapes/character recognition.

inkCanvas.InkPresenter.UnprocessedInput.PointerPressed += 
    UnprocessedInput_PointerPressed;
inkCanvas.InkPresenter.UnprocessedInput.PointerMoved += 
    UnprocessedInput_PointerMoved;
inkCanvas.InkPresenter.UnprocessedInput.PointerReleased += 
    UnprocessedInput_PointerReleased;

The "cut the lines" visual feedback

Let's start: When the user toggle 'on', we want to be in teh "cut the lines" mode:

We put the InputProcessingConfiguration.Mode to InkInputProcessingMode.None.
We go to UnprocessedInput by handling the three events.

private void ToggleSwitch_Toggled(object sender, RoutedEventArgs e)
{
	ToggleSwitch toggleSwitch = sender as ToggleSwitch;

	var p = inkCanvas.InkPresenter;
	if (toggleSwitch.IsOn == true)
	{
		// We are not in the inking or erasing mode 
		// ==> Inputs are redirected to UnprocessedInput
		// i.e. Our code will take care of the inking inputs
		p.InputProcessingConfiguration.Mode = InkInputProcessingMode.None;

		p.UnprocessedInput.PointerPressed += UnprocessedInput_PointerPressed;
		p.UnprocessedInput.PointerMoved += UnprocessedInput_PointerMoved;
		p.UnprocessedInput.PointerReleased += UnprocessedInput_PointerReleased;
	}
	else
	{
		// Go back to normal which is the inking mode
		// ==> We can draw lines on the InkCanvas
		inkCanvas.InkPresenter.InputProcessingConfiguration.Mode = InkInputProcessingMode.Inking;

		// Remove the handlers for UnprocessedInput
		p.UnprocessedInput.PointerPressed -= UnprocessedInput_PointerPressed;
		p.UnprocessedInput.PointerMoved -= UnprocessedInput_PointerMoved;
		p.UnprocessedInput.PointerReleased -= UnprocessedInput_PointerReleased;
	}
}

When the user starts to draw the 'cutting line', we are in the PointerPressed event handler. Here are the actions we do:

We create a XAML Polyline which allows us to 'draw' connected straight lines to display a dotted scissor line: lasso = new Polyline().
The Polyline will be a red dotted line like "- - -". This is what means the DoubleCollection() { 2, 2 } for the StrokeDashArray property.
The initial position of the finger/mouse/pen is used for the polyline: lasso.Points.Add(args.CurrentPoint.RawPosition);.
Lastly, the polyline is added to the canvas by using ShapesCanvas.Children.Add(lasso);.

// The lasso is used to cut the lines when we switch to UnprocessedInput
private Polyline lasso;

private void UnprocessedInput_PointerPressed(
    InkUnprocessedInput sender,
    Windows.UI.Core.PointerEventArgs args)
{
	lasso = new Polyline()
	{
		Stroke = new SolidColorBrush(Windows.UI.Colors.Red),
		StrokeThickness = 2,
		StrokeDashArray = new DoubleCollection() { 2, 2 }
	};

	lasso.Points.Add(args.CurrentPoint.RawPosition);
	ShapesCanvas.Children.Add(lasso);
}

For intermediate changes, when the user is drawing, it is pretty straightforward: We just add an additional point to the polyline:

private void UnprocessedInput_PointerMoved(
    InkUnprocessedInput sender, 
    Windows.UI.Core.PointerEventArgs args)
{
	lasso.Points.Add(args.CurrentPoint.RawPosition);
}

The "cut the lines" ending code

When the 'cutting line' drawn is finished, we have some work:

For each single XAML line of the canvas, we have to detect if it was cut or not. If the line was cut, we have to replace it by the two corresponding lines. This implies finding the intersection point and create the first resulting line from the origin to the intersection point and the second line for the intersection point to the end. Finally, we remove the original line which was cut.
Remove the 'cutting line' (the lasso variable) from the canvas because we do not need it anymore.

private void UnprocessedInput_PointerReleased(
    InkUnprocessedInput sender,
    Windows.UI.Core.PointerEventArgs args)
{
	lasso.Points.Add(args.CurrentPoint.RawPosition);
	
	// For each line of the Canvas, we look for an intersection
	// If any, we replace the line which has been cut by the two lines
	
	ShapesCanvas.Children.Remove(lasso);
}

Finding the intersections

To be honest, this part of the job was the most difficult. I first tried to make calculations with a sheet of paper and a pencil. I do not know if I would have been able to build an algorithm but I switched quickly to my second option which was do a search on Internet. And guess what! I found that a person named Rod Stephens were providing the exact code I need! Unbelievable! I contacted Rod and he said Yes: "Feel free to use the code and explanation". Thank you very much Rod! :call_me_hand:

You can find the original blog post of Rod at this url: Determine where two lines intersect in C# - http://csharphelper.com/blog/2014/08/determine-where-two-lines-intersect-in-c/.

Here is the function I used in my project.

p1, p2, p3, p4 are points with (X,Y) coordinates.
This function is used for each line of the canvas. The line goes from p1 to p2.
The 'cutting line' goes from p3 to p4.
segments_intersect is the boolean we get indicating if the line was cut or not.
intersection is the intersection point we will use to create the two new lines based on the line which was cut.

Note: There are extra parameters that we do not use in our sample application. You may need them based on your needs. So we chose to not change the signature of the function.

// Code provided by Rod Stephens
// at http://csharphelper.com/blog/2014/08/determine-where-two-lines-intersect-in-c/
// Find the point of intersection between
// the lines p1 --> p2 and p3 --> p4.
private void FindIntersection(
	PointF p1, PointF p2, PointF p3, PointF p4,
	out bool lines_intersect, out bool segments_intersect,
	out PointF intersection,
	out PointF close_p1, out PointF close_p2)
{
	// Get the segments' parameters.
	float dx12 = p2.X - p1.X;
	float dy12 = p2.Y - p1.Y;
	float dx34 = p4.X - p3.X;
	float dy34 = p4.Y - p3.Y;

	// Solve for t1 and t2
	float denominator = (dy12 * dx34 - dx12 * dy34);

	float t1 =
		((p1.X - p3.X) * dy34 + (p3.Y - p1.Y) * dx34)
			/ denominator;
	if (float.IsInfinity(t1))
	{
		// The lines are parallel (or close enough to it).
		lines_intersect = false;
		segments_intersect = false;
		intersection = new PointF(float.NaN, float.NaN);
		close_p1 = new PointF(float.NaN, float.NaN);
		close_p2 = new PointF(float.NaN, float.NaN);
		return;
	}
	lines_intersect = true;

	float t2 =
		((p3.X - p1.X) * dy12 + (p1.Y - p3.Y) * dx12)
			/ -denominator;

	// Find the point of intersection.
	intersection = new PointF(p1.X + dx12 * t1, p1.Y + dy12 * t1);

	// The segments intersect if t1 and t2 are between 0 and 1.
	segments_intersect =
		((t1 >= 0) && (t1 <= 1) &&
			(t2 >= 0) && (t2 <= 1));

	// Find the closest points on the segments.
	if (t1 < 0)
	{
		t1 = 0;
	}
	else if (t1 > 1)
	{
		t1 = 1;
	}

	if (t2 < 0)
	{
		t2 = 0;
	}
	else if (t2 > 1)
	{
		t2 = 1;
	}

	close_p1 = new PointF(p1.X + dx12 * t1, p1.Y + dy12 * t1);
	close_p2 = new PointF(p3.X + dx34 * t2, p3.Y + dy34 * t2);
}

The "cut the lines" ending code revealed

Let's put all the pieces together for effectively cutting all the lines which were intercepted.

Note: Of course you can do extra work to cut only one line and stop the UnprocessedInput code to check when a line is encoutered and cut.

In our application, we decided to cut all lines for which we have an intersection with the 'cutting line' (the lasso). Here are the steps:

Take notes about all the new lines we create. We use the linesToAdd list that we populate.
The foreach (Line line in ShapesCanvas.Children.OfType<Line>()) { } loop allows to perform the intersection check on every single XAML Line Shapes in the canvas.
The FindIntersection function is called for each XAML line.
We verify if there is an intersection with the boolean SegmentIntersection.
If yes, we call the CutTheLine function that we will describe later.
Finally, all the new lines are added to the canvas and we remove the lasso.

private void UnprocessedInput_PointerReleased(InkUnprocessedInput sender, Windows.UI.Core.PointerEventArgs args)
{
	lasso.Points.Add(args.CurrentPoint.RawPosition);

	List<Line> linesToAdd = new List<Line>();

	foreach (Line line in ShapesCanvas.Children.OfType<Line>())
	{
		bool LineIntersection = false;
		bool SegmentIntersection = false;
		PointF IntersectionPoint;
		PointF p2;
		PointF p3;
		FindIntersection(
			// The line in the canvas
			new PointF((float)line.X1, (float)line.Y1), 
			new PointF((float)line.X2, (float)line.Y2),
					
			// The 'cutting line'
			new PointF((float)lasso.Points.First().X, (float)lasso.Points.First().Y),
			new PointF((float)lasso.Points.Last().X, (float)lasso.Points.Last().Y), 
					
			out LineIntersection,

			// Indicate if there is an intersection
			out SegmentIntersection, 
					
			// The intersection point
			out IntersectionPoint, 
					
			out p2, out p3);
				
		if (SegmentIntersection)
		{
			List<Line> lines = CutTheLine(line, IntersectionPoint);
			linesToAdd.AddRange(lines);
		}
	}

	foreach (Line line in linesToAdd)
	{
		ShapesCanvas.Children.Add(line);
	}

	ShapesCanvas.Children.Remove(lasso);
}

Cutting the line

We have done the most complex work so far. Cutting the line is the easiest action to do when we have the original line and the intersection point:

The first new line goes from the origin of the original line to the intersection
The second new line goes from the intersection to the end of the original line

We just change the color for fun and to be able to distinguish the two resulting lines.

In an upcoming article, we will draw some 'anchors' to the line. It is another way to identify two distinct lines

private List<Line> CutTheLine(Line lineToCut, PointF intersection)
{
	List<Line> lines = new List<Line>();

	var line1 = new Line();
	line1.Stroke = new SolidColorBrush(Windows.UI.Colors.DarkOrange);
	line1.StrokeThickness = 3;
	line1.X1 = lineToCut.X1;
	line1.Y1 = lineToCut.Y1;
	line1.X2 = intersection.X;
	line1.Y2 = intersection.Y;

	var line2 = new Line();
	line2.Stroke = new SolidColorBrush(Windows.UI.Colors.DarkViolet);
	line2.StrokeThickness = 3;
	line2.X1 = intersection.X;
	line2.Y1 = intersection.Y;
	line2.X2 = lineToCut.X2;
	line2.Y2 = lineToCut.Y2;

	lines.Add(line1);
	lines.Add(line2);

	return lines;
}

F I N A L L Y, we made it! Yes! Yes! We M A D E it! =)

All the source code is available on GitHub - https://github.com/microsoft/Windows-AppConsult-Samples-UWP/

You want to see it in action? Look:

Window with the 'cut the lines' functionality

Wrapping up

UnprocessedInput is the key to be able to use the Inking inputs for performing advanced actions on strokes or XAML Shapes. We used it for 'cutting' lines but think about that you would be able to select shapes by intersecting them. We could imagine also putting points on a surface and when we press a extra button, we joint all the points to create a complex shape, etc..

@sbovo for the AppConsult team.

Inking series' articles

This article is part of a series exploring concepts about inking and XAML Shapes. Here are all links:

Use the UWP Inking platform as input for advanced scenarios
Handling zoom in Inking applications
Turning to the dark side of inking = UnprocessedInput ⇐ You are here
Free your mind: Start manipulating XAML Shapes
XAML Shapes manipulation level up

References

Source code for this article - https://github.com/microsoft/Windows-AppConsult-Samples-UWP/
ToogleSwitch control - https://docs.microsoft.com/en-us/uwp/api/windows.ui.xaml.controls.toggleswitch
InkPresenter.UnprocessedInput Property - https://docs.microsoft.com/en-us/uwp/api/windows.ui.input.inking.inkpresenter.unprocessedinput
InputProcessingConfiguration - https://docs.microsoft.com/en-us/uwp/api/windows.ui.input.inking.inkpresenter.inputprocessingconfigu...
InkUnprocessedInput.PointerPressed Event - https://docs.microsoft.com/en-us/uwp/api/windows.ui.input.inking.inkunprocessedinput.pointerpressed
InkUnprocessedInput.PointerMoved Event - https://docs.microsoft.com/en-us/uwp/api/windows.ui.input.inking.inkunprocessedinput.pointermoved
InkUnprocessedInput.PointerReleased Event - https://docs.microsoft.com/en-us/uwp/api/windows.ui.input.inking.inkunprocessedinput.pointerreleased
XAML Polyline - https://docs.microsoft.com/en-us/uwp/api/windows.ui.xaml.shapes.polyline
Determine where two lines intersect in C# (by Rod Stephens) - http://csharphelper.com/blog/2014/08/determine-where-two-lines-intersect-in-c/

Products (52)

Special Topics (28)

Video Hub (447)

Most Active Hubs