Hello. New here.
I have good news and bad news. The perfect solution for you would be an iRiver 795 coupled with a Giant Squid lavalier mic. For wind screen you can pick up a 4-pack at Radioshack for about $2. The iRiver 795 is about half the size of a cigarette pack so it fits easily in any pocket, and it has a “lock” switch to prevent anyone from shutting it off.
The bad news is that they stopped making the iRiver 795 about 3 years ago and they are no longer readily available on ebay. If you keep looking maybe you’ll get lucky and find one?