This will work:

    wp_set_object_terms( '2', wp_get_object_terms( '1', 'my_taxonomy' ), 'my_taxonomy',array("fields"=>"ids"));

**Explanation:** added this: `,array("fields"=>"ids")` to the attempt i wrote in the question to make the `wp_get_object_terms` return array of ids alone (what the `wp_set_object_terms` want to get.