The infinitive as object as in "He asked me to come in." & the to-infinitive as part of predicative as in 'The house of Jane was not easy to find' both act as noun. Both of these 'to come & to find' are these object?
 He asked me [to come in].
 Jane's house was not easy [to find].
No, the bracketed infinitival clauses are not objects.
 is a catenative construction where the infinitival clause to come in is catenative complement of asked. Me is the direct object of asked and the understood subject of to come in.
In  the infinitival clause to find is complement of the adjective easy -- they form the adjective phrase easy to find.