Interesting situation recently… We were creating a new web app in central admin and it was being provisioned on the application server (Which ran the WFE service for content indexing) but not the client facing WFEs.  Reboot of servers, IISRESET, it was starting to get a bit annoying.  Eventually came up with this process:

  1. Delete the newly created web application
  2. Stop SharePoint administration service from services.msc on all SharePoint servers
  3. run stsadm -o execadmsvcjobs
  4. Start SharePoint administration service from services.msc
  5. Create new web application

Web app should now show up on all servers running the web application server. If this does not work, then you can also trying clearing the SharePoint Configuration Cache, but it’s a bit more involved:

  1. Stop the Timer service. To do this, follow these steps:

    1. Click Start, point to Administrative Tools, and then click Services.
    2. Right-click SharePoint 2010 Timer, and then click Stop.
    3. Close the Services console.
  2. On the computer that is running Microsoft SharePoint Server 2010 and on which the Central Administration site is hosted, click Start, click Run, type explorer, and then press ENTER.
  3. In Windows Explorer, locate and then double-click the following folder:
  4. %SystemDrive%\ProgramData\Microsoft\SharePoint\Config\GUID
  5. Notes
    1. The %SystemDrive% system variable specifies the letter of the drive on which Windows is installed. By default, Windows is installed on drive C.
    2. The GUID placeholder specifies the GUID folder. There may be more than one of these.
    3. The ProgramData folder may be hidden. To view the hidden folder, follow these steps:
      1. On the Tools menu, click Folder Options.
      2. Click the View tab.
      3. In the Advanced settings list, click Show hidden files and folders under Hidden files and folders, and then click OK.
      4. You can also simply type this directly in the path if you do not want to show hidden files and folders.
  6. Back up the Cache.ini file. (Make a copy of it. DO NOT DELETE THIS FILE, Only the XML files in the next step)
  7. Delete all the XML configuration files in the GUID folder (DO NOTE DELETE THE FOLDER). Do this so that you can verify that the GUID folders content is replaced by new XML configuration files when the cache is rebuilt.
    Note When you empty the configuration cache in the GUID folder, make sure that you do NOT delete the GUID folder and the Cache.ini file that is located in the GUID folder.
  8. Double-click the Cache.ini file.
  9. On the Edit menu, click Select All.
  10. On the Edit menu, click Delete.
  11. Type 1, and then click Save on the File menu. (Basically when you are done, the only text in the config.ini file should be the number 1)
  12. On the File menu, click Exit.
  13. Start the Timer service. To do this, follow these steps:
    1. Click Start, point to Administrative Tools, and then click Services.
    2. Right-click SharePoint 2010 Timer, and then click Start.
    3. Close the Services console.
  14. Note The file system cache is re-created after you perform this procedure. Make sure that you perform this procedure on all servers in the server farm.
  15. Make sure that the Cache.ini file in the GUID folder now contains its previous value. For example, make sure that the value of the Cache.ini file is not 1.
  16. Check in the GUID folder to make sure that the xml files are repopulating. This may take a bit of time.