What is the advantage of using VSTP compared to RSTP and in which situations would I use VSTP? Thank you
I'd recommend taking a look at the Juniper Techpubs for EX switching with spanning tree. Here's an excerpt from the VSTP page:
Juniper Networks EX Series Ethernet Switches provide Layer 2 loop prevention through Spanning-Tree Protocol (STP), Rapid Spanning-Tree Protocol (RSTP), Multiple Spanning-Tree Protocol (MSTP), and VLAN Spanning-Tree Protocol (VSTP). The default factory configuration for EX Series switches uses RSTP. If you use VLANs, however, we recommend that you enable MSTP unless your network requires the device compatibility provided by VSTP. Switches configured to run VSTP automatically assign each VLAN to one spanning-tree instance that runs RSTP. While this approach is useful to optimize network usage in small networks with a limited number of VLANs, a VSTP configuration in networks with several hundred VLANs can overload switch CPUs. MSTP ensures that your network doesn’t slow down from the increased network traffic caused by hundreds of VLANs, each with its own spanning-tree instance.
There is a good comparison of these protocols on that linked page. However generally speaking, RSTP forces all logical L2 topologies (each VLAN) to use the same physical topology (the current stable spanning tree toplogy), which leaves one or more links dormant and unused. VSTP enables each logical topology (VLAN) to have it's own spanning tree root switch and stable physical topology by using a separate STP instance for each VLAN. While this allows the engineer greater capacity to balance traffic across all links, because VSTP creates a separate instance for each VLAN, CPU usage increases incrementally for each VLAN configured.
VSTP generally is only used for compatibility with non-MSTP speaking switches. MSTP allows for the same control in traffic distribution and link utilization without the CPU cost. VSTP can only be configured for up to 253 VLANs, and would be sending or interpreting one packet per second for each VLAN, whereas MSTP tacks on info about multiple VLANs onto the end of one packet (per configured MSTP instance).