Types of tests
First, note that "unit testing" in WordPress generally goes beyond unit testing, and is more like functional/integration testing. This is more of a technical difference, but if you understand the difference between these different things, you may sometimes be confused by how WordPress "unit" tests things.
When you ask what parts of the plugin you should test, ideally you should be testing all of it. But as to what parts you should unit (or integration) test, and what parts you should use acceptance tests for, is a different matter.
Currently, for my plugins, I use two different layers of tests: "unit" tests with PHPUnit, and acceptance tests run in the browser via WP Browser. The unit tests test internal logic, whereas the acceptance tests check that when a user uses your plugin's UI it actually works.
Test
The things to unit test are:
- code that modifies values in the database
- code that performs calculations
- code that contains other logic
Basically, any function that you pass a value(s) to and it does something like this with it, and/or returns some value to you.
To test this kind of code, you write a test that calls the function in question, passing in particular values, and then assert that the result that is returned is what you expect. Or, for functions that modify the database, you would assert that the database contains the values that are expected after the function has been called.
Don't test
Code that is difficult to unit test, and some would perhaps argue shouldn't be unit tested, would be code that is basically just dedicated to outputting stuff into the browser. That's where acceptance tests would come in.
Also, since you mention actions/filters, it sounds like you might be wondering whether you should test that a function is hooked to the correct action, or that a function you have hooked to a filter is actually applied when WordPress calls that filter. This may or may not be needed, depending on the action or filter in question. Generally, I don't test this sort of thing, as it usually is something that is better suited to functional/acceptance tests (though acceptance tests only test them indirectly).
Summary
So basically, unit test logic and database methods, acceptance test your plugin's actual UI. In order for this to work well, of course, you need to keep your template/output code mostly separate from your other logic, which is generally a good idea anyway.
In your case, you may find that acceptance tests are really more useful for your plugin, and unit tests wouldn't really be needed, since most everything would already be tested by the acceptance tests. However, having both would generally be the ideal.