Astrophysicist/Astronomer here: all the gravitational effect that *can* be happening *is* happening. Conservation of mass and energy dictates that all the mass in existence creates some amount of total sum of gravitational effect that remains unchanged whether it is distributed or condensed. So, as far as we understand, no amount of black holes will impact the expansion of the universe, because they wouldn't change the total net gravity of the universe.
HOWEVER.
We understand that the universe is expanding--that is, spacetime itself, which all matter moves within. The farther away an object is from us in spacetime, the faster it is moving away from us, meaning that there is a linear acceleration of spacetime expansion velocity positively proportional to distance (aka getting faster the farther away it is) when two objects are not locally bound by the effect due to gravity, which we call the Hubble Constant.
This implies that there exists region beyond the sphere of the observable universe where galaxies are located, but we will NEVER observe (the unobservable universe) because the spacetime of which they are a part is expanding faster than the speed of light (C), so light leaving them cannot move forward through spacetime because the distance between two points of spacetime in those areas is growing larger FASTER than light can speed from one point to the next.
Imagine walking up a downward escalator, but the faster you run to reach the top, the faster the treadmill moves downward, and it is ALWAYS moving your speed or FASTER. Thats what's happening to light in the most distant reaches just beyond the edges of the observable universe -- and we know this because we have PHYSICALLY MEASURED IT by measuring the velocities of faraway galaxies and found that they are all moving away from us in linear velocities proportional to their distances. Aka, if a galaxy is lcoated, say, 3 billion light years away, it is moving away from us at a specific velocity. We discovered this due to red shifting & blue blue shifting, AKA the exact same Doppler Effect that causes ambulances to sound weird as they pass us, except with light and electromagnetism rather than sound.
BUT
It gets even weirder: if we flew to the Andromeda Galaxy, 2.5 million light-years away from the Milky Way Galaxy, we would see 2.5 million light-years FARTHER in one direction than we can from the Milky Way, and 2.5 million light-years shorter in the opposite another direction. But some of the light in the *farther* direction will NEVER reach the Milky Way, because even though Andromeda is moving toward us through spacetime (due to our gravitational effect on each other), spacetime itself is expanding between the Milky Way and Andromeda, just fast enough to push that light to a standstill RELATIVE to the Milky Way's position in spacetime. The light *is* moving through spacetime, but with a net change in position of zero.
What this means is all points within spacetime have a DIFFERENT observable universe of the EXACT SAME RADIUS. No two points in space have the exact same view of the universe.
Finally, because all distant, gravitationally-unbound points of spacetime are accelerating away from us, eventually our view of the universe will darken as those distant points pass the speed of light, leaving only the 1,000 or so galaxies we are gravitationally bound to, the only ones that will remain until all the stars die and can't recycle, and black holes begin forming in the trillions of years thereafter, until there is only one black hole remaining, and it, too, dissipates away, and no light will ever shine again.
Yeah. Have your brain keep you up about THAT all night.
Came because of a photo of a marked up manuscript, staying for the intergalactic existential dread.