Floor mop holder with handle