If you look only at the Google requirement that new apps use API 26+ by August 1, then no, you don’t need to update. Those apps will be fine. Any new apps after August 1 have to support API level 26+. Updated apps have to support API 26+ at some point afterward, but existing apps do not have to be updated.
Now that said, Google has made changes to their “Vitals” program that tracks apps that don’t respond, crash, etc. Apps that crash too often or don’t respond can get down-ranked. This public release helps improve your apps with regards to Google Vitals, in particular, if you’re using ad plugins so you may want to update regardless.
Rob