Well, I think you are running the online demos for Aspose.Cells.GridWeb control which does show this issue. The reason is the online demos are using some older version of the product:
We recommend you to try our latest offline demos that do use our latest versions of the product (e.g Aspose.Cells.GridWeb v2.7.17.2000). I have tested it (using custom number formatting as per your attached screen shot), it works fine as per MS Excel.
Please download and try our latest offline demos for your complete reference: