Correct use of Addhandeler and RemoveHandeler

Hello there, ok I am trying to understand adding and removing event handelers.

In this very small example I have 1 button and 2 labels.

When the button is clicked for the first time label is loaded with “Hello” and label1 is loaded with “Hello World”.

When the button is clicked subsequently, label is loaded with “Hello” and label1 is loaded with “”.

<Window x:Class="MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
        xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
        xmlns:local="clr-namespace:Event_Test"
        mc:Ignorable="d"
        Title="MainWindow" Height="350" Width="525" Loaded="Window_Loaded"><Grid><Button x:Name="button" Content="Test" HorizontalAlignment="Left" Margin="86,37,0,0" VerticalAlignment="Top" Width="75"/><Label x:Name="label" Content="Label" HorizontalAlignment="Left" Margin="86,139,0,0" VerticalAlignment="Top" Width="194"/><Label x:Name="label1" Content="Label" HorizontalAlignment="Left" Margin="242,139,0,0" VerticalAlignment="Top"/></Grid></Window>

Class MainWindow



    Private Sub Window_Loaded(sender As Object, e As RoutedEventArgs)


        AddHandler button.Click, AddressOf WriteToLabel
        AddHandler button.Click, AddressOf WriteToLabel1



    End Sub

    Private Sub button_Click(sender As Object, e As RoutedEventArgs) Handles button.Click

    End Sub

    Private Sub WriteToLabel()
        label.Content = "Hello"
        If label1.Content = "Hello World" Then
            label1.Content = ""
        End If
    End Sub

    Private Sub WriteToLabel1()
        label1.Content = "Hello World"
        RemoveHandler button.Click, AddressOf WriteToLabel1
    End Sub


End Class

My question is –

In this line

RemoveHandler button.Click,AddressOf WriteToLabel1

The “WriteToLabel1” is underlined in green and has a message saying the removal has no effect.

What does this mean and how should I add and remove the handeler in this example?