GridSplitter with fixed background

I have a grid with 3 columns and a GridSplitter between each columns (in the column with horizontal alignment left). What I am trying to do is that the background of each column will stay in a fixed position while when moving the GridSplitter it will just show/hide parts of that column

XAML:

<Grid x:Name="MainGrid" SizeChanged="MainGrid_SizeChanged">
        <Grid.ColumnDefinitions>
            <ColumnDefinition x:Name="LeftColumn"/>
            <ColumnDefinition x:Name="CenterColumn"/>
            <ColumnDefinition x:Name="RightColumn"/>
        </Grid.ColumnDefinitions>
        <Image Stretch="Fill" Source="BG-Park.png" Grid.Column="0" Width="{Binding ElementName=MainGrid, Path=ActualWidth}" x:Name="LeftGrid" Panel.ZIndex="10"/>
        <Image Stretch="Fill" Source="Scenario2.png" Grid.Column="1" Width="{Binding ElementName=MainGrid, Path=ActualWidth}" x:Name="CenterGrid" Panel.ZIndex="5"/>
        <Image Stretch="Fill" Source="Scenario3.png" Grid.Column="2" Width="{Binding ElementName=MainGrid, Path=ActualWidth}" x:Name="RightGrid"/>
        <GridSplitter x:Name="CenterSplitter" Panel.ZIndex="20" Grid.Column="1" Width="3" VerticalAlignment="Stretch" Background="Black" HorizontalAlignment="Left" DragDelta="CenterSplitter_DragDelta"/>
        <GridSplitter x:Name="RightSplitter" Panel.ZIndex="21" Grid.Column="2" Width="3" VerticalAlignment="Stretch" Background="Black" HorizontalAlignment="Left" DragDelta="RightSplitter_DragDelta"/>
    </Grid>

Code behind:

        private void MainGrid_SizeChanged(object sender, SizeChangedEventArgs e)
        {
            CenterGrid.Margin = new Thickness((-1) * LeftColumn.ActualWidth, 0, 0, 0);
            RightGrid.Margin = new Thickness((-1) * (LeftColumn.ActualWidth + CenterColumn.ActualWidth), 0, 0, 0);
        }

        private void CenterSplitter_DragDelta(object sender, System.Windows.Controls.Primitives.DragDeltaEventArgs e)
        {
            if ((LeftColumn.ActualWidth > 0) && (CenterColumn.ActualWidth > CenterSplitter.ActualWidth))
            {
                double lLeft = Math.Min(CenterGrid.Margin.Left - e.HorizontalChange, 0);
                lLeft = Math.Max(lLeft, (-1) * (CenterGrid.ActualWidth - CenterSplitter.ActualWidth));
                CenterGrid.Margin = new Thickness(lLeft, 0, 0, 0);
            }
        }

        private void RightSplitter_DragDelta(object sender, System.Windows.Controls.Primitives.DragDeltaEventArgs e)
        {
            if ((CenterColumn.ActualWidth > 0) && (RightColumn.ActualWidth > 0))
            {
                double lLeft = Math.Min(RightGrid.Margin.Left - e.HorizontalChange, 0);
                lLeft = Math.Max(lLeft, (-1) * (RightGrid.ActualWidth - RightSplitter.ActualWidth));
                RightGrid.Margin = new Thickness(lLeft, 0, 0, 0);
            }
        }

This works as expected except for when the GridSplitters touch each other or the edge of the grid and than separate, than the image starts to move around until you stop the dragging of the GridSplitter, if you start again it will continue properly but from the last position it stopped.

Any ideas why this is and how I can fix it?

I tried calculating the Margin according to the ActualWidth of the columns but that caused the image to jump every time the GridSplitter moves.