basically, when turning, the square tail corners will dig into the wave, providing more grip and drive, but requiring more force from the rider.
The round / round pin will transition more smoothly from rail to rail.
On these question of board design, I found that Rusty blog posts to be the most informative: lots on insights in a very short read:
www.surfline.com/blogs/talking-design-the-board-blog-with-rusty-preisendorfer_26649/On the tails:
www.surfline.com/blogs/talking-design-the-board-blog-with-rusty-preisendorfer/rusty-breaks-down-the-lower-third-of-surfboards-from-squash_30118/www.surfline.com/blogs/talking-design-the-board-blog-with-rusty-preisendorfer/history-of-tails-part-2--rusty-talks-greenough-shortboard-r_30849/quoting the master:
Squash or rounded square: This is probably the most common small/medium wave tail shape. It offers a nice blend of area and curve. It became the go-to tail shape when three fins took over the world in the early 1980s. There were wide squash tails built early in the shortboard era. The problem with wide tails on single fins was that the wider the tail (block) the deeper the fin needed to be. The deeper the fin, the more the frontal drag. With three fins, it really enabled designers to go wider and shorter, moving area aft because of the hold and drive three fins brought to the game. The tail shape provides lift, support, and drive. The balance of release and smoothness or continuity depends on how rounded the corners are.Round tails: Uber-smooth turns. Make a clay model of a hollow wave, use a wire to slice it along the speed line in the barrel, and the cross section will probably look something like this. Or Google conic sections. This tail is a clean, natural curve that fits the pocket. It's a great tail for a performance shortboard, good in head-high to double-overhead waves. Also, a good option for next one-up type shapes.