Hi Dean,
set security policy default-policy permit all will only affect security policy only .
Firewall Filter will work fine as configured.
SRX in Packet mode will restain you with security related configuration only,
Firewall Filter and SSH will work fine.
Regards
rparthi
Please Mark My Solution Accepted if it Helped, Kudos are Appreciated Too