Plunderphonics

In the Plunderphonics paper, Oswald says “as curious tourists should we not be able to take our own snapshots through the crowd rather than be restricted to the official souvenir postcards and programmers”. As I understand it, he is saying that we should have the right as individuals to interact with music from our own unique angles and thereby remix the culture surrounding us according to our inclinations. In this way, listeners no longer are just passive. Instead, they too become active in the process of music making much like a person who is enjoying the architecture of a beautiful city through the lens of his camera.
However, I am still a bit confused as to what exactly constitutes plagiarism when sampling music. A person can take a picture of the Eiffel Tower in his own unique fashion and not worry about such restrictions, but in the world of music, the rules seem to be different and not well defined. At least from what I can understand at this point.
The Wikipedia page on John Oswald quotes him as saying, “if creativity is a field, copyright is the fence”. I found that to be a very succinct and memorable way of explaining the problem that Plunderphonics runs into.