Like others have said before - both are great devices, and have their own USPs and a good list of features. Point is you need to think over how useful the features of each of these devices be to your usage habits. IME, Passport is mind-blowing, unlike any other device out there. As a device it can do certain things Note 4 can't (physical keyboard, keyboard scrolling, wider screen etc). Likewise, Note 4 does certain things Passport can't (the S-pen I love, multiwindow etc.).
I'd still prefer the Passport if I had to choose one because of the OS. At the end of the day that's what it comes to. One is always composing, reading, switching between- messages. And BB10 trumps every other OS in this regard.
As one small example, consider this: you need to share a link with your friend on fb chat. You have this link in a previous chat with someone else on Whatsapp. To do this on a Note 4 (any Android) , you'll first have to swipe into the app tray where Whatsapp resides, tap to open Whatsapp, select the specific chat from the chat list, copy the link, press home button to either select the fb app from running apps or go the app tray again and tap open the fb app/messenger, select the chat with your friend and then paste your link. Doing the same thing on Passport would need you to simply swipe right to enter hub, tap on the Whatsapp chat in the hub, copy the link, swipe back to hub, select the fb chat above, paste and done. There could be many more such use cases. The OS is more efficient in terms of getting things done. Try composing a SMS on Android and BB10, you'll note the difference.
What I am saying is, Note 4 will give you access to a wider range of apps, and Passport will let you do things more efficiently, while also allowing you to run Android apps for whatever isn't available natively. Is the experience of using Android apps same on Passport? For me, yes. I use around 15-20 Android apps, and all of them work as well as they do on Nexus 5. Of course, there are few which don't work, but they are not on the list of Android apps I use.
Posted via CB10