You can take a look at this slide: NSX-v 6.4 - Load Balancing ToI
slide #76
The pool defined in the Virtual Server would be something else.
Do you do any HTTPS offload or anything? What's the HTTPS settings on LB and on Server?

# Check if host is "app1.xyz.com" (case insensitive)
acl host_app1 hdr(Host) -i app1.xyz.com
# Check if host is "app2.xyz.com" (case insensitive)
acl host_app2 hdr(Host) -i app2.xyz.com
# Check if host is "app3*" (case insensitive)
acl host_any_app3 hdr_beg(host) -i app3
# Use specific pool for each hostname
use_backend pool_app1 if host_app1
use_backend pool_app2 if host_app2
use_backend pool_app3 if host_any_app3
#for everything else, use the pool defined in the Virtual Server
#Some other random examples of AppRules
# Some ACLs to check URL from Client request
acl url_a path_beg -i /a
acl url_t path_beg -i /t
acl url_test_nopersist path -i /test_nopersist.php
# Some ACLs to Cookie from Client request
acl cookie_dimi1 hdr_sub(cookie) dimi1=1
acl cookie_sitet hdr_sub(cookie) cookie_sitet
# Select special LB_pool based on ACL
use_backend pool_vipa if url_a
use_backend pool_vipt if url_t
use_backend pool_cookie1 if cookie_dimi1
# Select special LB_pool based on multiple ACLs
use_backend pool_multiple if url_t cookie_dimi1
# Inject a new Cookie based on ACL
rspadd Set-Cookie:\ inject1=1 if url_t
# Delete specific cookie based on ACL
reqidel ^Cookie:.*cookie_sitet= if url_test_nopersist
# Rewrite GET without "/app1"
acl app1 path_beg -i /app1
reqirep ^GET\ /app1/(.*)\ HTTP/(.*) GET\ /\1\ HTTP/\2 if app1