Libya would have been lost, BUT (and this is a MAJOR but) Russia would have faced more German heat, earlier. Enough to take one of Leningrad, Stalingrad, or Moscow?
Enough to make a difference?
The Siege of Sevastopol also known as the Defence of Sevastopol (Russian: Оборона Севастополя, transliteration: Oborona Sevastopolya) or simply the Battle of Sevastopol (German: Schlacht um Sewastopol) was a military battle and a siege that took place on the Eastern Front of the Second World War. The campaign was fought by the Axis powers of Germany, Romania, and Italy against the Soviet Union for control of Sevastopol, a port in the Crimea on the Black Sea. On 22 June 1941 the Axis invaded the Soviet Union during Operation Barbarossa. Axis land forces reached the Crimea in the autumn of 1941 and overran most of the area. The only objective not in Axis hands was Sevastopol. Several attempts were made to secure the city in October and November 1941. A major attack was planned for late November, but heavy rains delayed the Axis attack until 17 December 1941. Under the command of Erich von Manstein, Axis forces were unable to capture Sevastopol during this first operation. Soviet forces launched an amphibious landing on the Crimean peninsula at Kerch in December 1941 to relieve the siege and force the Axis to divert forces to defend their gains. The operation saved Sevastopol for the time being, but the bridgehead in the eastern Crimea was eliminated in May 1942.
After the failure of their first assault on Sevastopol, the Axis opted to conduct siege warfare until the middle of 1942, at which point they attacked the encircled Soviet forces by land, sea, and air. On 2 June 1942, the Axis began this operation, codenamed Störfang (Sturgeon Catch). The Soviet Red Army and Black Sea Fleet held out for weeks under intense Axis bombardment. The German Air Force (Luftwaffe) played a vital part in the siege. The Luftwaffe made up for a shortage of Axis artillery, providing highly effective aerial bombardment in support of the ground forces. Finally, on 4 July 1942, the remaining Soviet forces surrendered and the Axis seized the port. Both sides had suffered considerable losses during the siege and attack.
With the Soviet forces neutralised, the Axis refocused their attention on the major summer campaign of that year, Operation Blue and their advance to the Caucasus oil fields.
Did Stalin do enough to prevent Sevastopol from being captured or did he just let it fall?
He used it in the same manner as Hitler ( incorrectly) used Stalingrad to engage the bulk of German forces that are not bearing down against Moscow.