Problem while accessing user space memory in a trap handler in Linux

I am trying to make the invalid opcode trap handler able to print out the offending opcode. The platform is an x86-64 machine. I am doing it by rewriting the handler as

dotraplinkage void do_invalid_op(struct pt_regs *regs, long error_code) {
    // print the offending opcode
    pr_info("offending opcode is 0x%02X\n", *(uint8_t *)(regs->ip));

    // the original routine
    do_error_trap(regs, error_code, "invalid opcode", X86_TRAP_UD, SIGILL, ILL_ILLOPN, IP);
}

However when I execute an invalid instruction (a hand-crafted invalid instruction), I get the error of

[   24.114923] BUG: unable to handle kernel paging request at 0000000000400096

Here I have checked that 0x0000000000400096 is indeed the invalid instruction so there's no problem with the rip I get. It is frustrating to me because I don't think there's a context switch but somehow I cannot access the user space memory. I am aware that get_user and copy_from_user could be used for this but they can sleep so I cannot use them inside a trap handler.

Is there anything I am doing wrong? What should I do to implement such behaviour?