I never exercise courtesy when it comes to "right-of-way" scenarios; it insinuates continued traffic confusion and could result in a fatal collision.
Just don't be the guy in the second-from-right lane preventing the driver in a right lane - which happens to end just up ahead - from changing lanes. Backing off a smidge and letting the driver in the right lane do the lane change will keep things flowing.
Best one I've seen involved three cars who flew past a transport truck that was in the ending-right-lane scenario. Truck's signal was on the whole time; drivers to the left ignored it; the end of the right lane in this case was enforced by a bridge abutment. Eventually the truck just had to do the lane change and forced the last car to attempt the pass to slam on the brakes. Totally foreseeable and avoidable.
Just because you have the right of way doesn't mean you should take it ...